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Abstract 

The  objective  of  this  research  was  to  analyze  the 
adequacy  of  using  commercial  manuals  and  to  identify  the 
correct  acquisition  process  to  obtaining  adequate  commercial 
manuals . 

A  questionnal;-e  was  developed  and  a  survey  was 
accomplished.  The  survey  was  distributed  to  Deputy  Program 
Managers  for  Logistics  (DPHL)  and  Air  Logistics  Centers  that 
utilized  commercial  manuals.  The  most  significant  result 
indicated  that  the  commercial  manuals  that  were  determined 
to  be  adequate,  actually  had  not  been  thoroughly  reviewed. 
This  resulted  in  additional  funding  requirements  for  manual 
supplements  and  system  interim  contract  support.  Other 
significant  issues  included:  the  importance  of  early 
involvement  in  the  acquisition  program  phase;  and,  the  DID 
and  the  SOW  as  effective  contract  mediums. 

Recommendations  to  improve  the  effectiveness  of  the 
commercial  manual  review  included: 

1.  Establishing  a  universal  specification  that  would 
be  used  in  all  contracts. 

2.  Establishing  verification  procedures  for  commercial 
manuals  on  equipment  that  is  integrated  into  a  system.' 
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3.  Establishing  an  internal  technical  writing 
department  that  will  do  the  changes  to  commercial  manuals.' 

4.  Establishing  the  use  of  similar  hardware  (CD 
readers)  to  the  industrial  contractors. 
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THE  ADEQUACY  OF  COMMERCIAL  MANUALS  FOR 
INTERMEDIATE  AND  DEPOT  LEVEL  MAINTENANCE 


I .  Introduction 

Problem  Statement 

The  Deputy  Assistant  Secretary  of  Defense  (Logistics) 
is  concerned  with  shortening  the  acquisition  cycle  and 
reducing  the  cost  of  repairing  Non-Development 
Items/Commercial-Of f-The-Shelf  Equipment  (NDI/COTS) . 
Currently,  the  repair  of  NDI/COTS  equipment  can  be 
accomplished  by  Contractor  Logistics  Support  (CLS) 

(permanent  contractor  maintenance  support) ;  Interim 
Contractor  Support  (ICS)  (temporary  contractor  maintenance 
support) ;  or  Organic  Support  (AF  depot  maintenance) .  The 
type  of  support  will  depend  upon  the  economics,  security, 
and  war  readiness  capability  requirements  of  each  respective 
system. 

Specific  Problem 

The  Air  Force  (AF)  spends  millions  of  dollars  on  weapon 
systems  that  require  "adequate”  technical  documentation, 
especially  for  organic  depot  support.  This  data  should  be 
thorough  enough  that  operations  and  maintenance  can  occur  in 
a  timely  and  cost  efficient  manner.  A  major  concern  is  the 
incomplete  publications  procured  to  support  weapon  systems. 
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Currently,  technical  publications  are  being  procured  that  do 
not  reflect  current  hardware  configuration,  nor  do  they 
consider  equipment  standardization.  Configuration  management 
is  a  procedure  for  applying  technical  and  administrative 
direction  and  surveillance  to  (a)  identify  and  document  the 
functional  and  physical  characteristics  of  an  item  or 
system,  (b)  control  any  changes  to  such  characteristics  and 
(c)  record  and  report  the  change,  process,  and 
implementation  status  [3:13].  Since  the  government  does  not 
have  configuration  management  control  in  commercial  manual 
acquisitions ,  the  contractor  may  decide  to  update  equipment 
in  order  to  advance  with  technology  or  to  make  a  one-time 
fix.  In  some  instances,  these  changes  modify  internal 
configurations  but  do  not  get  documented  into  the  technical 
docvunentation.  Permanent  changes  are  documented  when  they 
affect  the  contractor's  entire  inventory.  These  one-time 
changes  reduce  equipment  standardization  (each  like  system 
being  identical) ,  add  to  equipment  repair  time,  and 
sometimes  add  to  reverse  engineering  cost. 


Research  Objective 

Questions  were  developed  to  test  the  hypothesis  that 
commercial  manuals  are  not  always  the  most  cost  effective 
method  when  considering  the  cost  of  buying  adequate 
documentation  vs  the  cost  of  developing  documentation,  the 
cost  of  reverse  engineering,  and  the  increase  in  maintenance 
turnaround  time.  To  support  this  hypothesis,  a  questionnaire 


was  developed  and  then  presented  to  Deputy  Program  Managers 
for  Logistics  and  depot  maintenance  personnel  within  AFSC 
and  AFLC.  The  personnel  were  selected  by  systems  known  to 
have  NDI  equipment.  The  developed  survey  used  to  collect 
data  to  support  this  hypothesis  is  attached  as  Appendix  A. 

Background 

The  government  requires  accurate  data  in  order  to 
maintain  and  operate  their  weapon  systems.  Since  the 
1950 's,  the  DoD  has  been  scrutinizing  the  issue  of 
commercial  equipment  and  proprietary  rights.  In  1964,  DoD 
adopted  a  policy  stating  that  they  would  honor  rights  to 
technical  data  pertaining  to  items,  components,  or  processes 
"developed  at  private  expense"  if  contractors  would  deliver 
such  data  to  the  government  for  use  in  operating  and 
maintaining  military  hardware  [7:51]. 

The  remainder  of  this  background  section  will  involve 
three  discussions  concerning  commercial  equipment  and 
commercial  manuals.  The  first  will  identify  the  events 
leading  up  to  the  Packard  Commission's  recommendation  to 
utilize  commercial  equipment.  The  Packard  Commission's  Blue 
Ribbon  study  led  by  David  Packard  was  the  first  in  depth 
study  on  commercial  equipment  usage.  The  discussion  will 
include  the  Packard  Commission's  recommendation,  the 
Competition  Act  of  1984,  and  the  latest  public  laws 
concerning  proprietary  data  rights;  along  with  how  the 
balance  between  the  contractor  and  the  government  concerning 
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data  rights  was  disrupted  by  these  actions.  The  second 
discussion  will  identify  current  problems  associated  with 
use  of  commercial  manuals.  The  final  discussion  will 
emphasize  the  concern  of  the  dependency  being  developed  on 
contractor  support. 

Prior  to  the  Packard  Commission's  recommendation,  many 
events  occurred.  Following  is  a  brief  summary  of  those 
events : 


In  1972  the  commission  on  government  procurement 
(COGP)  had  a  shift  in  philosophy  and  recommended 
placing  greater  reliance  on  privately  developed  off- 
the-shelf  products  and  established  commercial 
distribution  channels  to  support  those  products 
[2:2]  . 

In  May  1976  the  COGP  recommendations  became 
policies  in  memorandum  format.  According  to  Klein 
and  East's  1976  memo,  "the  government  should 
purchase  commercial  off  the  shelf  (COTS)  products 
whenever  they  would  adequately  serve  the 
government's  requirements,  provided  the  products 
have  established  commercial  market  acceptability 
[2:2-3] ." 

In  1977  the  Commercial  Commodity  Acquisition 
Program  (CCAP) ,  a  Defense  Logistics  Agency, 
documented  cases  of  commercial  acquisitions 
accomplished  without  detailed  specification.  Also, 
the  Commercial  Item  Support  Program  (CISP)  began  a 
review  to  minimize  the  level  of  centralized 
management  required  for  potentially  commercial 
distribution  channels  [2:3]. 

In  1978  both  the  CCAP  and  the  CISP  were  included 
in  DoD  Directive  5000.37  thus  developing  Commercial 
Item  Acquisition  and  Distribution  of  Commercial 
Products  (ADCOP)  [2:5]. 

In  early  1980  the  Army  began  work  based  on  the 
ADCOP  concepts  [2:6]. 

April  1986,  the  Packard  Commission's  Blue  Ribbon 
report  was  presented  to  the  President. 
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In  April  1986,  the  Blue  Ribbon  Cominission  on  Defense 

Management  reported  that  rather  than  relying  on  excessively 

rigid  military  specifications,  DoD  should  make  greater  use 

of  components,  systems  and  services  available  from  the 

private  sector.  It  should  develop  new  or  custom-made  items 

only  when  it  had  been  established  that  those  readily 

available  commercial  items  were  clearly  inadequate  to  meet 

military  requirements.  Their  report  stated: 

"The  DoD  should  make  greater  use  of  components, 
systems,  and  services  available  "off  the  shelf," 
developing  new  or  custom-made  items  only  when  "off- 
the-shelf"  ones  do  not  meet  military  needs." 

"Federal  law  and  DoD  regulations  should  provide  for 
substantially  increased  use  of  commercial-style 
competition,  emphasizing  quality  and  established 
performance  as  a  price,  particularly  for  research 
and  development  and  for  professional  services 
[9:60] ." 

The  commission's  main  concern  was  to  reduce  over  all 
program  cost  and  expedite  the  "fielding"  of  acquisition 
systems.  With  the  acceptance  of  these  recommendations, 
three-to-f ive  years  of  the  development  time  can  be 
transferred  to  usable  time.  It  also  aids  in  reducing 
research  and  development  costs  of  systems  already  available. 
Packard  cites  two  methods  attributing  to  the  time  and  cost 
problem — "user  pull  and  technology  push."  [9:45]  "User 
pull"  is  where  the  users  (who  may  or  may  not  be  technically 
knowledgeable)  define  the  system  with  all  the  "wish  list" 
features.  "Technology  pull"  is  when  the  government  wants 
the  most  up  to  date,  state  of  the  art  technology  possible. 
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Both  methods  direct  programs  toward  high-cost  and  long  life 
cycles  from  the  beginning. 

The  commission's  concern  is  strengthened  by  General 
Richard  H.  Thompson's  speech  on  shortening  the  acquisition 
strategy: 

"We  can  no  longer  afford  to  design  equipment  to  meet 
the  entire  realm  of  environmental  possibilities, 
especially  if  the  environment  will  encounter  those 
conditions  only  10  to  15  percent  of  the  time.  We 
must  start  designing  for  the  "expected"  rather  than 
routinely  designing  for  the  "worse  case.”  We  must 
question  and  tailor  specifications  that  don't  make 
good  sense  in  meeting  product  requirements  [5:35]." 

The  Defense  Authorization  Act  of  1984,  Section  907  of 
Public  Law  (PL)  99-661,  Preference  for  Nondevelopmental 
Items,  was  passed.  This  requires  the  DoD  to  state 
requirements  for  supplies  in  terms  of  functions  to  be 
performed,  performance  required,  essential  physical 
characteristics,  and  definition  of  those  requirements  so 
that  non-development  items  (NDI)  can  be  procured  to  fulfill 
them  [2:2-3].  This  law  amended  Chapter  137  of  Title  10 
United  States  Codes  to  require  maximum  utilization  for  NDI. 

Now  that  the  law  prefers  the  acquisition  of 
nondevelopment  items,  the  DoD  still  feels  the  necessity  to 
have  data  that  normally  is  not  sold  with  the  commercial 
equipment  and  to  the  customer  for  competitive  procurement 
purposes  as  indicated  in  one  of  the  recommendations  by  the 
Procurement  Round  Table,  a  follow-on  committee  to  the 
Packard  Commission  [7:53].  This  has  impacted  the  contractors 
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willingness  to  give  out  the  information.  As  the  Packard 
Commission  points  out, 

"DoO  can  use  its  unique  position  to  enhance  U.  S. 
industry's  worldwide  technological  position;  or 
unwittingly,  through  the  pursuit  of  other  shorter 
term  goals,  to  reduce  incentive  for  developing  new 
technology;  or,  even  worse,  make  commercially 
valuable  technology  available  to  international 
competitors  [9:64]." 

Since  the  use  of  nondevelopmental  equipment  is  becoming  more 
prevalent,  the  use  of  the  "old-ways"  or  boilerplating  the 
contractor's  requirements  are  not  adequate. 

While  commercial  equipment  went  through  a  decade  of 
events,  a  battle  began  and  continues  today  over  data  rights. 
When  the  government  acquired  mostly  MIL-SPEC  systems,  the 
contractor  provided  nearly  all  of  the  required  data.  The 
only  exception  was  information  developed  at  private 
industry's  own  expense.  As  the  events  led  to  utilizing  more 
commercial  equipment,  the  contractors  began  to  feel 
threatened.  If  they  gave  up  this  information,  they  would  no 
longer  have  a  competitive  edge  on  the  market.  It  equates 
out  to  losing  military  secrets  to  the  enemy.  Through  an 
unwritten  code,  the  contractors  struck  a  balance  on 
commercial  data  rights  to  the  government.  The  contractors 
would  provide  proprietary  rights  to  the  government  as  long 
as  it  was  not  used  outside  of  the  intended  maintenance  use. 
However,  this  pushed  the  government  into  high  cost  systems 
because  it  forced  reprocurement  buys  from  sole  sources.  In 
order  to  combat  these  high  costs  through  competition,  the 


Contracting  Act  of  1984  was  issued.  It  made  reprocurement 
competitive,  however,  the  balance  between  goveriuaent  and 
contractor  technical  data  rights  was  broken  [7:51].  Further 
research  into  the  current  data  rights  requirements  was  then 
accomplished  by  the  Procurement  Round  Table  in  1987  [10:24] . 

Current  data  rights  requirements  are  P.L.  98-525  and 
P.L.  98-577  of  1984,  and  P.L.  99-661  of  1984.  It  must  be 
noted  that  it  was  not  until  May  12,  1987  that  the  DoD  FAR 
Supplement  (DFAR)  was  updated  to  reflect  these  laws.  It  is 
DFAR  27.4  [7:51-53  and  4:51-55]. 

Between  the  passing  of  the  three  laws  and  the  updating 
of  the  DFAR,  two  more  acts  were  passed  to  accelerate  DoD 
enactment.  After  many  attempts  to  define  the  technical  data 
rights  requirements,  the  1987  DoD  Authorization  Act  was 
enacted  on  January  16,  1987.  Here  congress  (1)  detailed  the 
specific  provisions  to  be  included  in  the  new  DoD  clause, 

(2)  instructed  the  DoD  to  consider  how  to  address  rights  in 
jointly  funded  development  data,  and  (3)  set  a  new  deadline 
for  the  promulgation  of  the  new  regulation.  Then  on  April 
16,  1987,  the  DAR  council  issued  final  regulations 
implementing  section  953  of  public  law  99-500,  the  Defense 
Acquisition  Improvement  Act  [11:36]. 

All  of  this  activity  and  time  lag  attributes  to  the 
confusion  for  both  the  government  and  the  contractor.  More 
attempts  have  been  made  and  are  being  made  to  clarify  this 
situation.  Currently,  the  Procurement  Round  Table  has 
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recoimnended  a  new  policy  with  five  elements,  the  five 
elements  are: 

1.  A  single  regulation; 

2.  Separate  technical  data  for  computer  programs; 

3.  Protecting  commercial  rights  in  technical  data; 

4.  Compensation  for  licensing  of  competitors;  and, 

5.  Controlling  the  techniques  used  to  obtain 
competition  [7:52-53]. 

Using  nondevelopmental  equipment  has  not  been  without 
other  problems .  The  technical  data  rights  are  only  one 
complication.  When  the  government  procures  NDI  equipment, 
they  lose  control  of  configuration,  cost,  manufacturing, 
repair  parts  replacement  (supportability) ,  and  maintenance 
(maintainability).  Overall,  the  question  must  be  addressed 
concerning  the  AF  system  supportability  and  availability 
posture  when  "combat"  logistics  become  necessary. 

Some  of  the  preceding  conclusions  in  the  above 
paragraph  are  from  the  DoD  Logistics  Support  Analysis  Office 
(LSAO)  study  accomplished  in  1986.  The  LSAO  conclusion  to 
the  use  of  nondevelopment  items  is  that  it  is  not  always 
beneficial.  They  state, 

"NDI  acquisitions  will  cause  item  proliferation  if 
sufficient  technical  data  is  not  acquired  [6:iv]." 

"Some  areas  of  counter  productivity  are  "repair  part 
standardization,  NATO  standardization  (or  any 
standardization  for  that  matter) ,  the  competition 
breakdown  program,  and  service  policies  to  procure 
all  technical  data  and  have  National  Stock  Numbers 
(NSN)  for  all  spares  before  fielding  the  system 
[6:7]  ." 
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For  systems  with  a  life  cycle  exceeding  five  years, 

LSAO  is  concerned  that  the  contractor  will  have  changed 
parts  or  configuration  of  the  sub-assemblies  or  the  entire 
system.  In  some  instances  the  contractor  may  decide  it  is 
not  economical  to  continue  making  the  system  and  quit  making 
the  parts.  When  this  happens,  the  government  loses 
equipment  standardization  and  support  for  the  item.  Thus, 
causing  the  government  to  find  a  replacement  for  the 
equipment  [12] . 

Improper  technical  data  can  cause  a  lack  of 
configuration  control  which  impacts  the  government's 
commercial  manuals  and  its  systems.  This  adverse  effect  is 
illustrated  through  the  AN/RQA-17  monitor.  This  program  has 
organic  depot  support.  While  the  monitor  was  going  down  the 
production  line,  the  manufacturer  would  substitute 
components  on  printed  circuit  boards  if  they  ran  out  of  the 
correct  component.  The  substitute  components  would  keep  the 
equipment  electronically  compatible;  however,  it  created 
trouble  for  the  maintainer.  The  AF  bought  the  recommended 
spares,  however,  these  spares  only  worked  on  some  of  the 
serial  numbered  parts.  As  a  result,  the  Air  Force  had  to 
build  a  matrix  showing  the  serial  number  of  each  unit  cross 
referenced  to  the  layout  of  the  components  and  the 
electronic  value  of  the  components.  In  one  instance  the 
manufacturer  ran  out  of  a  47k  ohm  resistor.  They  replaced 
it  with  a  33k  ohm  resistor  with  another  resistor  added 


M 

* 

i 

M 

J 

I 


J 


10 


elsewhere  to  compensate  for  the  substitution.  The 
contractor  did  not  document  this  in  the  commercial  manual . 

When  this  board  with  the  substitution  was  sent  to  the 
maintainer,  the  resistor  was  burnt  beyond  recognition.  The 
maintainer  had  followed  the  instructions  in  the  commercial 
manual,  unaware  of  the  substitution,  only  to  find  that  the 
repair  failed.  Through  trial  and  error  the  maintainer  found 
the  appropriate  resistor.  This  took  up  needless  time  and 
money  [12]  .  What  if  this  had  been  in  a  battlefield 
situation? 

There  is  concern  about  the  cost  of  obtaining 
"adequate”  technical  data.  Data  which  is  adequate  to 
maintain  and  operate  equipment,  may  be  too  costly.  The  high 
price  tag  for  purchasing  "proprietary  data"  is  used  by  the 
contractor  as  a  protection  device.  Contractors  are  afraid 
trade  secrets  will  be  lost  to  other  companies.  Therefore, 
they  must  protect  their  competitive  edge.  These  high  price 
tags  must  be  compared  to  the  cost  of  contractor  maintenance 
for  the  life  of  the  system  (CLS) .  In  the  short  run,  CLS  is 
more  cost  effective  than  buying  data  rights.  However,  can 
we  afford  it  in  the  long  run? 

After  the  government  has  bought  and  fielded  the  system, 
the  contractor  may  decide  that  it  is  no  longer  profitable  to 
make  the  same  system  or  to  make  spares  for  the  old  models. 
They  may  decide  it  is  more  profitable  to  update  the  system. 
In  any  case,  the  government  is  negatively  impacted  because 
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commercial  manuals  are  often  not  updated.  They  may  be 
supplemented,  but  not  in  enough  detail. 

One  example  of  the  commercial  manual  not  accurately 
portraying  the  equipment  is  the  AN/GIQ-18  monitor.  The 
AN/GIQ-18  demonstrates  the  impact  of  lost  configuration 
control.  In  this  monitor  there  are  eight  different 
configurations  under  the  same  part  number  [12] .  This 
creates  numerous  problems  for  the  maintainer.  The  problems 
range  from  what  needs  correcting  to  what  part  number  is 
needed  to  repair  it. 

Another  impact  from  procuring  nondevelopment  equipment 
is  the  procurement  of  replacement  parts  through  sole  source 
efforts.  This  happens  even  though  the  government  has 
stressed  competitive  buys.  Since  requisitioning  occurs  from 
the  part  numbers  in  the  commercial  manual,  the  reprocurement 
of  the  repair  parts  is  frequently  from  a  sole  source  [l:i]. 
This  is  because  the  commercial  developers  assign  their  own 
part  n;imber,  which  is  also  their  internal  inventory  number. 
These  inventory  numbers  appear  in  the  commercial  manuals . 
Even  if  a  part  is  a  suitable  substitute  for  the  old  system, 
the  part  number  needs  to  appear  in  the  commercial  manual  for 
the  maintenance  person  to  requisition  the  part. 

Because  of  the  impacts  previously  mentioned,  the 
government  is  in  a  position  of  dependence  on  the  commercial 
developer  to  maintain  spares  support  for  the  life  cycle  of 
the  equipment  [l:i].  In  cases  where  there  will  be  organic 
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support,  the  coitunercial  manuals  often  are  not  ready  when  the 
system  is  fielded.  An  example  is  the  acquisition  of  the 
AN/USM-488,  general  purpose  oscilloscope.  The 
oscilloscope's  performance  range  is  direct  current  0  to 
lOOMHz.  The  purchase  was  for  8000  items.  Due  to  quality 
problems  in  the  commercial  manuals,  there  was  approximately 
a  ten-month  delay  in  their  availability  [2:app  a] .  The 
government  had  to  depend  on  the  contractor’s  interim  support 
(ICS)  which  cost  additional  money. 

For  weapon  systems  that  have  CLS,  are  the  contractors 
willing  to  do  annual  training  for  combat  situations?  Are 
they  willing  to  take  up  arms?  If  they  are  not,  is  it  fair 
to  have  the  AF  maintainer  put  in  a  position  to  defend  the 
contractor  on  the  battlefield?  Or  is  it  feasible  to  have 
the  AF  maintainer  open  the  equipment  for  the  first  time  and 
repair  it  quickly?  Not  only  are  they  untrained  for  the 
particular  "black-box",  they  are  under  combat  duress. 

A  situation  that  illustrates  this  position  occurred  in 
World  War  II  on  Wake  Island.  There  were  400  Marines  and 
1100  civilian  construction  contractors  on  the  island  at  the 
time  the  Japanese  bombed  Pearl  Harbor,  Hawaii.  When  the 
Japanese  attacked  Wake  Island,  the  Marines  had  the 
additional  duty  to  protect  the  civilian  contractors.  In 
their  strategic  and  tactical  planning  they  had  to  take  them 
into  consideration.  The  Marines  could  not  force  the 
contractors  to  fight,  however,  about  half  of  them 
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volunteered.  There  were  not  enough  weapons,  so  they  helped 
with  the  trench  digging  etc.  Then  as  Marines  fell,  they 
took  up  their  arms  and  fought  back.  The  other  half  of  the 
contractors  felt  that  if  they  did  not  fight  the  Japanese 
would  treat  them  differently  and  that  they  would  be  safe  if 
the  island  were  overtaken.  The  island  was  overtaken,  and 
the  contractors  that  had  fought  were  sent  with  the  marines 
to  a  prisoner  of  war  camp.  The  remaining  contractors  were 
used  as  a  labor  force  on  the  island  and  then  shot  when  they 
were  no  longer  needed  [8] . 

This  story  illustrates  the  additional  responsibility 
the  contractor  may  place  on  the  soldier  during  times  of 
conflict.  It  raises  the  concern  about  becoming  critically 
dependent  on  a  civilian  force  during  peacetime. 

This  NDI/COTS  review  indicates  that  the  use  of 
commercial  manuals  and  equipment  can  negatively  impact,  (1) 
the  supportability  (spares)  caused  by  the  lack  of 
configuration  management;  (2)  the  availability  (delay  in 
start  date) ;  and  (3)  the  maintainability  (not  being  able  to 
repair  due  to  improper  procedures  and  the  lack  of  spares) 
[6:iv;  7:36] . 


J 

j 

I 


I 


14 


II .  Methodolo 


Justification 

Since  the  Packard  Commission's  study,  emphasis  has  been 
placed  on  the  procurement  of  NDI/COTS  equipment. 

Technical  orders  are  used  by  all  DoD  services  for  the 
operations  and  maintenance  of  weapon  systems  at  all  levels 
(organizational,  intermediate,  and  depot).  If  the  users  or 
the  maintainers  do  not  have  complete  technical 
documentation,  the  systems  productivity  can  be  hindered. 
Also,  extensive  damage  could  occur  due  to  inappropriate 
operating  and  maintaining  procedures. 

By  utilizing  NDI/COTS  equipment,  the  cost  of  developing 
and  researching  weapon  systems  can  be  reduced?  however,  not 
eliminated.  For  100  percent  NDI/COTS  equipment,  the  cost  is 
absorbed  by  the  contractor  since  the  R  &  D  occurred  prior  to 
DoD  consideration. 

Also,  the  use  of  NDI/COTS  equipment  is  suppose  to 
reduce  the  cost  of  technical  documentation  (drawings, 
manuals,  etc.)  However,  the  DoD  must  consider  whether  or 
not  these  commercial  manuals  meet  the  requirements  to 
fulfill  operational  and  maintenance  obligations.  In  some 
instances,  the  commercial  manuals  are  so  incomplete  that  the 
DoD  becomes  dependent  on  the  contractor  to  support  the 
equipment  after  fielding. 

As  a  result  of  the  Packard  Commission's  1986  report, 
the  Deputy  Assistant  Secretary  of  Defense  (logistics) 
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requested  studies  be  accomplished  to  re-evaluate  "the 
implications  of  NDI  Systems  Acquisition"  and  to  provide 
further  study  areas  [letter  3  Jul  1986] .  One  area  requiring 
further  research  is  NDI/COTS  manuals.  Within  this  area, 
commercial  manuals  are  impacted  if  equipment  is  not 
standardized  or  the  manuals  are  not  updated  to  equipment 
changes.  The  operator  and  maintainer  requires  manuals, 
commercial  or  military,  that  correspond  to  the  same  model  as 
being  operated  or  maintained. 

When  the  DoD  procures  military  manuals  (MIL-SPEC  TO) , 
specific  requirements  are  put  on  contract  prior  to  the 
manual's  development.  Then,  throughout  the  manual’s 
development,  the  services  do  in-process-reviews  (IPR) , 
verifications,  testings,  and  pre-publication  reviews. 
However,  when  the  DoD  procures  a  commercial  manual,  the 
manual  has  already  been  developed  and  published.  In  some 
instances,  the  service  may  review  the  commercial  manual 
prior  to  the  contract  or  use  the  manual  for  testing  prior  to 
initial  operating  capability  (IOC).  In  the  latter  use,  the 
logistical  consequence  is  realized  late  in  the  acquisition 
cycle,  and  interim  contract  support  must  be  used. 

The  surveys  and  interviews  that  will  be  done  in  support 
of  this  thesis  will  demonstrate  the  adequacies  and 
inadequacies  of  commercial  manuals  and  the  developing 
dependency  for  Contract  Logistics  Support  (CLS) . 
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Instrument 


The  questionnaire  (Appendix  A)  was  designed  into  seven 
sections.  Each  section  provides  an  inter-/intra- 
relationship  to  commercial  manuals. 

Section  one  refers  to  the  relationship  between 
commercial  manuals  and  the  overall  system.  The  questions 
address  the  program  phase,  the  maintenance  concept  (organic 
or  contract,)  and  the  type  of  weapon  system  being  surveyed. 
These  questions  will  be  used  to  find  if  the  elements  of  the 
program  phase  influence  whether  or  not  commercial  manuals 
should  or  should  not  be  used  through  a  frequency  table. 

Section  two  refers  to  the  commercial  manuals  and  their 
relationship  to  the  system  (section  one)  and  to  MIL-SPEC 
manuals  (section  three) .  Currently,  the  Air  Force  does  not 
have  a  requirement  to  review  the  commercial  manuals  prior  to 
the  contract,  and  do  not  have  a  set  of  in-process-reviews 
that  will  be  accomplished  after  the  contract  award.  The 
only  formalized  review  stems  from  MIL-M-7298C.  This 
military  standard  provides  a  checklist  of  requirements  that 
the  contractor  and  the  Technical  Order  Management  Agency 
(TOMA)  reviews.  A  desktop  review  without  equipment 
availability  is  usually  accomplished  against  this  checklist. 

In  section  two,  an  attempt  is  being  made  to  identify  a 
correlation  between  the  review  procedures  and  the  quality 
and  adequacy  of  commercial  manuals  from  the  Air  Force's 
perspective.  The  analysis  will  compare  variables  from 


section  two  to  the  same  variables  of  section  three,  and 
variables  from  section  two  to  variable  88.  Variable  88 
identifies  the  commercial  manuals  and  service  bulletins 
adequacy  for  use  by  level  5  and  7  technicians . 

Section  three  refers  to  the  MIL-SPEC  manuals, 
manuals  that  are  developed  during  the  acquisition  phase, 
then  reviewed  and  tested  prior  to  the  equipment  fielding. 
Questions  in  section  three  were  used  to  compare  the 
governments  approach  in  procuring  MIL-SPEC  manuals  to  their 
approach  in  procuring  commercial  manuals.  The  questions  are 
identical  to  section  two,  commercial  manuals. 

Section  four  refers  to  the  contractual  requirements 
(ordering,  reviewing,  and  approving)  for  commercial  manuals. 
Also,  this  section  is  used  to  identify  if  additional  funding 
is  required  after  contract  award.  Both  analyses  will  be 
presented  through  a  frequency  table. 

Section  five  refers  to  the  users  perception  of  the 
adequacy  of  the  finished  commercial  manual.  These  questions 
respond  to  the  issue  of  equipment  standardization  and 
configurations  identified  in  the  commercial  manual.  If 
either  issue  is  identified,  the  person  has  been  requested  to 
address  how  the  issue (s)  were  resolved  and  the  amount  of 
additional  time  that  was  required  to  resolve  the  problem ( s) . 

Section  six  refers  to  the  need  and  the  adequacy  of 
service  bulletins,  notices  from  the  contractor  of  equipment 
changes  or  equipment  safety  factors.  First,  the  questions 
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establish  whether  or  not  the  conunercial  manuals,  without 
supplementation,  are  usable  to  the  same  level  as  the  MIL- 
SPEC  manuals.  If  they  are  not,  the  person  must  identify  how 
much  supplementing  is  required  to  raise  the  manuals  to  a 
usable  level.  Second,  the  questions  ask  whether  or  not  the 
users  are  receiving  the  updated  service  bulletins. 

Section  seven  allows  the  participants  to  respond,  in 
their  own  words,  to  the  problems  and/or  advantages  of  using 
commercial  manuals.  These  statements  are  encouraged  to  be 
open,  not  necessarily  along  the  "thought-stream"  of  the 
questionnaire.  Some  of  the  responses  may  be  issues  that 
will  be  open  for  further  research. 

Sample/Population 

There  are  two  methods  of  research  that  will  be  used, 
interviewing  and  surveying.  Contacts  were  selected  from 
weapon  system  program  offices  (SPO) ,  depot  maintenance 
personnel,  and  deputy  program  managers  for  logistics  (DPML) 
for  NDI/COTS  systems.  The  discussion  during  the  interviews 
will  be  a  closer  evaluation  of  the  problems;  whereas,  the 
survey  questionnaire  identifies  if  a  model  for  adequate 
commercial  manuals  can  be  built. 

The  surveyed  were  selected  from  a  complete  list  of 
DPMLs  provided  by  the  AMIS  office  at  Wright-Patterson  AFB. 
Each  office  at  ASD  and  Space  Division  was  called  and  asked 
if  their  weapon  system  involved  commercial  equipment  and  if 
they  would  participate  in  the  survey.  However,  the  DPML 
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from  Electronics  Systems  Division  were  selected  at  random. 
Approximately  forty  responded  positively.  Of  these  forty, 
they  were  asked  to  provide  the  office  symbols  of  the 
corresponding  maintenance  facilities  and  air  logistics 
centers  (ALC)  requirement  branches.  In  total,  sixty  surveys 
were  distributed.  One  week  after  the  surveys  were  to  be 
returned,  follow-up  phone  calls  were  made  to  gather  the 
remaining  surveys.  Only  thirty-five  were  returned. 

The  interviewed  were  selected  through  individuals  who 
indicated  that  they  wanted  to  talk  in  person  during  the 
previously  mentioned  phone  calls;  and,  individuals  known  to 
have  experience  with  commercial  manuals.  Also,  in  order  to 
perceive  the  commercial  companies  preparation  procedures  in 
developing  commercial  manuals,  representatives  from  General 
Electric  were  interviewed. 

Data  Collection  Plan 

The  survey  data  collection  consist  of  59  questions. 
These  questions  were  categorized  into  92  independent 
variables.  The  independent  variables  are  listed  below.  The 
first  number  is  the  variable  number.  The  second  number, 
which  appears  in  parenthesis  after  the  variable  title,  is 
the  value  assigned  for  a  specific  response  during  analysis. 
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Section  One. 

1  *  Program  Phase 

Conceptual  Phase  (1) 

Demonstration/Validation  Phase  (2) 

Full  Scale  Development  Phase  (3) 

Production  Phase  (4) 

Deployment  Phase  (5) 

2  =  Maintenance  Concept 

2- level  (Organization,  Depot)  (1) 

3- level  (Organization,  Intermediate,  Depot)  (2) 

Other  (0) 

3  =  Is  the  depot  level  maintenance  done  through  organic 

support? 

Yes  (1)  No  (0) 

4  =  Is  the  depot  level  maintenance  done  through  contract 

support? 

Yes  (1) 

No  (0) 

5  =  Is  the  intermediate  maintenance  done  through  organic 

support? 

Yes  (1) 

No  (0) 

6  =  Is  the  intermediate  maintenance  done  through  contract 

support? 

Yes  (1) 

No  (0) 

7  =  How  many  years  will  interim  contract  support  be  used? 

Less  than  1  year  (.5) 

1-3  years  (2) 

4- 6  years  (5) 

7-10  years  (8.5) 
over  10  years  (11) 

8  =  End  item  quantity  of  the  system 

1-5  (3) 

6-15  (10.5) 

16-30  (23) 

31-50  (40.5) 
over  50  (51) 
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Equipment  type 
Communications  Equipment  (1) 

Support  Equipment  (2} 

Aircraft  (3) 

Computer  Resources  (4) 

Space  System  (5) 

Other  (6) 

10  s  Was  a  cost  comparison  done  between  the  commercial 

manual  and  supplemental  cost  versus  MIL-STD  technical 
order  development  cost? 

Yes  (1) 

No  (0) 


Section  Two. 

11  s  Are  the  commercial  manuals  used  at  the  intermediate 

facility? 

Yes  (1) 

No  (0) 

12  »  Are  the  commercial  manuals  used  at  the  depot  facility? 

Yes  (1) 

No  (0) 

13  »  Commercial  manuals  make-up  what  percent  of  the  total 

system  manuals? 

Less  then  10%  (.5) 

10- 30%  (.2) 

31-50%  (.405) 

Over  50%  (.51) 

14  =  Did  the  contract  require  the  contractor  to  review  the 

commercial  manuals  in  accordance  with  MIL-M-7298C? 

Yes  (1) 

No  (0) 

15  =  Was  the  contractor  or  the  government  required  to 

review  the  commercial  manuals  prior  to  contract  award? 
Yes  (1) 

No  (0) 

♦Variables  16  -  43  were  only  answered  if  variable  15  was 
yes . 

16  =  Number  of  days  allotted  to  review  the  manuals  prior  to 

contract  award. 

0-10  days  (5) 

11- 30  days  (20.5) 

31-60  days  (45.5) 
over  60  days  (61) 


22 


Government  representatives  that  reviewed  the  commercial 
manuals 

17  =  Integrating  Contractor 

Yes  (1) 

No  (0) 

18  ®  Logistics  Support  Contractor 

Yes  (1) 

No  (0) 

19  =  Program  Manager 

Yes  (1) 

No  (0) 

20  =  Logistics  Manager 

Yes  (1) 

No  (0) 

21  =  Equipment  Specialist 

Yes  (1) 

No  (0) 

22  =  Engineer 

Yes  (1) 

No  (0) 

23  =  Technical  Order  Specialist 

Yes  (1) 

No  (0) 

24  =  Maintenance  Technician 

Yes  (1) 

No  (0) 

25  =«  Using  Command 

Yes  (1) 

No  (0) 

Purpose  of  the  commercial  manual  review 

26  =  Maintenance  Procedures 

Yes  (1) 

No  (0) 

27  =  Data  rights  Restrictions 

Yes  (1) 

No  (0) 

28  =  Part  Number  Accuracy 

Yes  (1) 

No  (0) 

29  =  Safety  Warning  Notices 

Yes  (1) 

No  (0) 

30  =  Hardware  Accuracy 

Yes  (1) 

No  (0) 


Type  of  quality  check  used  for  the  commercial  manuals 

31  a  Validation 

Yes  (1) 

No  (0) 

32  «  Verification 

Yes  (1) 

No  (0) 

33  =  In  Process  Review 

Yes  (1) 

No  (0) 

34  =  If  a  verification,  validation,  or  in-process  review 

was  accomplished,  were  there  differences  between  the 
original  operational  procedures  and  the  actual 
findings? 

Yes  (2) 

No  (1) 

Do  not  know  (0) 

Location  of  the  commercial  manual  review 

35  =  Contractor's  Facility 

Yes  (1) 

No  (0) 

36  =  Air  Force  Facility 

Yes  (1) 

No  (0) 

37  =  Was  the  actual  equipment  available  during  the 

commercial  manual  review? 

Yes  (1) 

No  (0) 

38  =  Was  the  equipment  used  during  the  commercial  manual 

review? 

Yes  (1) 

No  (0) 

39  =  Reading  grade  level  of  the  commercial  manuals 

Ninth  grade  (1) 

Twelfth  grade  (2) 

Fourteenth  grade  (3) 

Other  (4) 

No  requirement  (0) 

40  =  Average  number  of  volumes  per  commercial  manual 

1  (1) 

2  (2) 

3  (3) 

4  (4) 

5  or  more  (5) 
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41  »  Are  detailed  diagrams  or  graphics  used  to  show  the 

maintenance  procedures  step  by  step? 

Yes  (1) 

No  (0) 

42  »  Does  an  illustrated  parts  breakdown  come  with  the 

commercial  manuals? 

Yes  (1) 

No  (0) 

43  «  Do  the  illustrated  parts  breakdown  diagrams  cross 

reference  each  other  and  the  text? 

Yes  (1) 

No  (0) 


Section  Three. 

44  =  Are  the  MIL-SPEC  manuals  used  at  the  intermediate 

facility? 

Yes  (1) 

No  ( 0 ) 

45  =  Are  the  MIL-SPEC  manuals  used  at  the  depot  facility? 

Yes  (1) 

No  (0) 

46  =  MIL-SPEC  manuals  make-up  what  percent  of  the  total 

system  manuals? 

Less  then  10%  (.05) 

10-30%  (.20) 

31-50%  (.405) 

Over  50%  (.51) 

47  =  Did  the  contract  require  the  contractor  to  review  the 

MIL-SPEC  manuals  in  accordance  with  MIL-M-38784? 

Yes  (1) 

No  (0) 

Type  of  quality  check  used  for  the  MIL-SPEC  manuals 

48  =  Validation 

Yes  (1) 

No  (0) 

49  =  Verification 

Yes  (1) 

No  (0) 

50  =  In  Process  Review 

Yes  (1) 

No  (0) 
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Government  representatives  that  reviewed  the  MIL-SPEC 
manuals 

51  *  Integrating  Contractor 

Yes  (1) 

No  (0) 

52  *  Logistics  Support  Contractor 

Yes  (1) 

No  (0) 

53  *  Program  Manager 

Yes  (1) 

No  (0) 

54  =  Logistics  Manager 

Yes  (1) 

No  (0) 

55  =  Equipment  Specialist 

Yes  (1) 

No  (0) 

56  =  Engineer 

Yes  (1) 

No  (0) 

57  =  Technical  Order  Specialist 

Yes  (1) 

No  (0) 

58  =  Maintenance  Technician 

Yes  (1) 

No  (0) 

59  =  Using  Command 

Yes  (1) 

No  (0) 

Purpose  of  the  MIL-SPEC  manual  review 

60  =  Maintenance  Procedures 

Yes  (1) 

No  (0) 

61  =  Data  rights  Restrictions 

Yes  (1) 

No  (0) 

62  =  Part  Number  Accuracy 

Yes  (1) 

No  (0) 

63  =  Safety  Warning  Notices 

Yes  (1) 

No  (0) 

64  *  Hardware  Accuracy 

Yes  (1) 

No  (0) 


i 


I 


■ 
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65  =  If  a  verification,  validation,  or  in-process  review 

was  accomplished,  were  there  differences  between  the 
original  operational  procedures  and  the  actual 
findings? 

Yes  (2) 

No  (1) 

None  accomplished  (0) 

Location  of  the  MIL-SPEC  manual  review 

66  a  Contractor's  Facility 

Yes  (1) 

No  (0) 

67  =  Air  Force  Facility 

Yes  (1) 

No  (0) 

68  =  Was  the  actual  equipment  available  during  the 

commercial  manual  review? 

Yes  (1) 

No  (0) 

69  =  Was  the  equipment  used  during  the  MIL-SPEC  manual 

review? 

Yes  (1) 

No  ( 0 ) 

70  =  Number  of  days  allotted  to  review  the  manuals  prior  to 

contract  award. 

0-10  days  (5) 

11-30  days  (20.5) 

31-60  days  (45.5) 
over  60  days  (61) 

71  =  Reading  grade  level  of  the  MIL-SPEC  manuals 

Ninth  grade  (1) 

Twelfth  grade  (2) 

Fourteenth  grade  (3) 

Other  (4) 

No  requirement  (0) 

72  *  Average  number  of  volumes  per  MIL-SPEC  manual 

1  (1) 

2  (2) 

3  (3) 

4  (4) 

5  or  more  (5) 

73  =  Are  detailed  diagrams  or  graphics  used  to  show  the 

maintenance  procedures  step  by  step? 

Yes  (1) 

No  (0) 
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74  =  Does  an  illustrated  parts  breakdown  come  with  the 

commercial  manuals? 

Yes  (1) 

No  (0) 

75  =  Do  the  illustrated  parts  breakdown  diagrams  cross 

reference  each  other  and  the  text? 

Yes  (1) 

No  (0) 


Section  Four. 

Commercial  manual  contract  requirements 

76  =  TMCR  86-01 

Yes  (1) 

No  (0) 

77  =  Data  Item  Descriptions 

Yes  (1) 

No  (0) 

78  =  CFE/CFAE  Notices 

Yes  (1) 

No  (0) 

79  =  AFADs 

Yes  (1) 

No  (0) 

80  «  SOW 

Yes  (1) 

No  (0) 

81  =  Did  the  commercial  manuals  require  additional  funding 

above  the  original  contract  projection? 

Yes  (1) 

No  (0) 


Section  Five. 

82  =  How  did  the  program  office  or  maintenance  facility 

handle  the  standardization  of  parts  in  commercial 
systems? 

Submitted  TO  change  (2) 

Went  back  to  the  contractor  to  correct  (1) 

Other  (0) 

83  =  Did  the  contractor  charge  for  corrections  required  in 

the  commercial  manuals? 

Yes  (1) 

No  (0) 
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84  =  What  caused  the  lack  of  standardization? 

The  maintenance  facility  (2) 

The  contractor  facility  (1) 

Both  (3) 

Other  (0) 

85  =  Were  there  any  cases  of  the  actual  equipment 

configuration  not  matching  the  technical  documentation 
during  subsequent  repair  activities? 

Yes  (1) 

No  (0) 

86  =  Number  of  mismatches  found  in  the  commercial  manuals 

0-2  (1) 

3-5  (4) 

6-8  (7) 

9-11  (10) 
over  11  (13) 

87  =  Once  a  problem  was  solved,  was  a  supplement  or  TO 

change  accomplished? 

Yes  (1) 

No  (0) 


Section  Six. 

88  =  Are  the  commercial  manuals  and  service  bulletins 

adequate  for  use  by  military  5  and  7  level 
technicians  in  repairing  the  equipment  to  the  same 
levels  the  AF  maintains  MIL-STD  equipment? 

Yes  (1) 

No  (0) 

89  =  To  what  extent  does  the  commercial  documentation 

require  supplementation? 

Under  5%  (.025) 

6-10%  (.08) 

11-20%  (.155) 

21-30%  (.255) 

Over  30%  (.31) 

90  =  Are  sufficient,  in-depth  documentation  that  complies 

with  the  system's  maintenance  concept  being  received? 
Yes  (1) 

No  (0) 

91  =  Was  the  contractor  required  to  provide  updates  and 

changes  with  commercial  equipment  modifications? 

Yes  (1) 

No  (0) 
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92  =  Are  all  the  users  getting  the  service  bulletins  and 
interim  safety  supplements? 

Yes  (1) 

No  (0) 

Statistical  Tests 

After  the  interviews  and  surveys  have  been 
accomplished,  the  data  will  require  some  statistical 
testing.  Sections  two  and  three  will  require  an  analysis  of 
variance  to  be  accomplished.  This  will  indicate  if  there  is 
a  procedure  or  review  difference  between  the  commercial 
manuals  and  the  MIL-SPEC  manuals  that  is  significant. 
Sections  one  and  five  will  be  used  for  a  regression  analysis 
to  analyze  the  correlation  between  the  type  of  maintenance 
(organic  or  contract)  and  the  commercial  manuals  shortfalls. 
Next,  sections  two  and  four  will  be  used  in  a  regression 
analysis.  This  analysis  will  be  used  to  indicate  which 
factors  are  more  significant  to  the  success  of  adequate 
commercial  manuals.  All  of  the  data  will  be  used  to 
identify  the  successful  procedures  that  have  been  used  and 
have  averted  problems. 

The  most  significant  hurdle  will  be  assuring  the 
interviewed  and  the  surveyed  complete  anonymity  so  that 
factual  data  can  be  collected. 
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Description  of  the  Survey 

The  survey  consisted  of  sixty  questions  regarding  the 
adequacy  of  commercial  manuals.  It  separated  into  six 
sections.  Each  section,  with  section  one  being  the 
exception,  was  directed  towards  an  element  that  may  or  may 
not  impact  the  quality  of  the  manual  the  Air  Force  receives 
as  a  deliverable  product.  Section  one,  the  system,  was  used 
to  establish  whether  or  not  the  program  being  surveyed  had 
reached  the  deployment  stage.  As  noted  in  chapter  two,  the 
six  sections  are:  the  system,  the  commercial  manual,  the 
MIL-SPEC  manual,  the  contract  requirements,  the  technician’s 
response  to  the  commercial  manual,  and  the  personal 
responses  of  the  survey  participants.  Appendix  A  contains  a 
copy  of  the  survey  and  the  response  percentages  for  each  of 
the  survey  questions.  The  survey  did  not  deviate  from  the 
original  design  set  up  in  the  methodology.  However,  the 
survey  failed  to  require  the  participants  to  respond  to  a 
set  of  questions  regarding  the  adequacy  of  the  MIL-M-7298C 
review.  The  questions  should  have  been  similar  to  the  set 
asked  of  the  pre-contract  award  review  (variables  16-43) . 
This  exclusion  caused  the  statistical  test  to  omit  a 
comparison  between  "in-process"  reviews  and  adequate 
commercial  manuals  at  the  time  of  deployment.  Also,  the 
survey  failed  to  have  the  participants  identify  whether  or 
not  the  MIL-SPEC  manuals  were  adequate  for  use  by  a  5  or  7 
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level  maintenance  technician.  This  oversight  caused  the 
statistical  tesc  comparing  the  review  process  differences  of 
commercial  manuals  and  MIL-SPEC  manuals  to  be  omitted. 

Recording  of  Findings 

The  format  of  Appendix  A  is  identical  to  the  survey 
questionnaire,  with  the  exception  of  the  last  two  columns. 
The  first  of  these  columns  identifies  the  number  of 
responses  for  each  variable  element;  and  the  last  column  is 
the  overall  percentage  for  the  variable  element.  The 
percentages  for  questions  1-15  and  44  -  92  are  based  on 
thirty-five  responses.  The  percentages  for  question  16  -  43 
are  based  on  fifteen  responses.  Question  16  -  43  were  only 
to  be  answered  if  question  15  was  responded  to  with  a  yes. 
Also,  some  questions  were  not  answered  by  all  of  the 
participants.  In  these  cases,  ”No  response”  was  added  and 
totaled.  The  raw  data  used  in  the  statistical  test  are 
contained  in  Appendix  B. 

Following  the  statistical  test  against  the  survey,  a 
summary  of  the  oral  interview  with  General  Electric  (GE)  is 
provided.  (Specific  questions  and  responses  are  contained 
in  Appendix  D.) 

Statistical  Tests 

Statistical  percentage  comparisons  were  accomplished  on 
the  dependent  variable  (X88} ,  the  commercial  manuals 
usefulness  by  a  5  or  7  level  technician,  using  the  list  of 
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independent  variables  shown  in  Table  I.  The  list  is  divided 
into  five  groups.  Group  one  contains  the  general  elements 
concerning  the  surveyed  system.  Group  two  represents  the 
job  titles  of  the  pre-contract  reviewers.  Group  three 
represents  the  elements  used  for  review.  Group  four 
represents  the  contract  elements  that  were  used  to  put  the 
commercial  manuals  on  contract.  Finally,  group  five 
contains  the  elements  used  to  analyze  the  cause  of 
additional  costs.  Since  the  percentage  of  not  adequate 
responses  were  larger  than  the  adequate  responses  by  20% 
under  variable  X88,  this  study  assumed  that  the  commercial 
manuals  were  not  adequate  for  a  5  or  7  level  technician  at 
the  time  of  deployment.  In  the  few  cases  where  the 
commercial  manuals  were  adequate,  the  study  analyzed  the 
results  to  find  what  was  being  done  correctly.  However,  the 
analysis  confirmed  that  the  inadequate  manuals  were  reviewed 
properly,  and  that  the  adequate  manuals  were  not.  The  data 
to  support  these  statements  will  be  developed  in  subsequent 
paragraphs . 


Comparison  Analysis 

Table  II  compares  the  independent  variable,  identified 
in  Table  1,  of  adequate  manuals  to  the  inadequate  manuals. 
In  total,  there  were  thirty-five  observations  from  the 
survey.  For  the  adequate  manuals,  there  were  fourteen  "yes 
responses  from  variable  X88.  The  remaining  twenty-one  "no" 


TABLE  I 


LIST  OF  INDEPENDENT  VARIABLES 
Group  One 


XI . Program  Phase 

X3 .  Depot/Organic  Support 

X4 .  Depot/Contract  Support 

X5 .  Intermediate/Organic  Support 

X6 .  Intermediate/Contract  Support 

X9 . Program  Type 

X14  .  Reviewed  against  MIL-M-7293C 

X15  .  Pre-Contract  Review  Accomplished 

Group  Two 

X17  .  Integrating  Contract 

X18  .  Logistics  Support  Contractor 

X19  .  Program  Manager 

X20  .  Logistics  Manager 

X21  .  Equipment  Specialist 

X22 . Engineer 

X23  .  Technical  Order  Specialist 

X24  .  Maintenance  Technician 

X25  .  Using  Command  Representative 

Group  Three 

X26  .  Maintenance  Procedures 

X27  .  Data  Rights  Restrictions 

X28 . Part  Number  Accuracy 

X29  .  Safety  Warning  Notice 

X30  .  Hardware  Accuracy 

Group  Four 

X76 . TMCR  86-01 

X77 . Data  Item  Description 

X78  .  CFE/CFAE  Notices 

X79 . AFAD 

X80 . Statement  of  Work 

Group  Five 

X81  .  Additional  Funding  Required 

X86 . Number  of  Mismatches  Found 


TABLE  II 


ADEQUATE  VERSUS  NON-ADEQUATE 


Independent 

ADEQUATE* 

"Yes" 

NON-ADEQUATE** 

"Yes" 

OVERALL*** 

Variable 

Response 

% 

Response 

% 

% 

XI  (deployment) 

6 

42.8 

3 

14.2 

25.7 

(production) 

2 

14.2 

3 

14.2 

14.2 

(FSD) 

5 

35.7 

5 

23.8 

28.6 

X3 . 

8 

57.1 

11 

52.4 

54.3 

X4 . 

8 

57.1 

13 

61.9 

60.0 

X5 . 

10 

71.4 

8 

38.1 

51.4 

X6 . 

6 

42.8 

6 

28.5 

34.3 

X9  (Comm  Elect) 

2 

14.2 

3 

14.2 

14.2 

(Supp  Equip) 

5 

35.7 

5 

23.8 

28.6 

(Aircraft)  . 

4 

28.5 

5 

23.8 

25.7 

(Computers) . 

1 

7.1 

1 

4.7 

5.7 

(Space) .  .  . 

1 

7.1 

5 

47.6 

17.1 

X14 . 

9 

64.3 

13 

61.9 

62.8 

X15 . 

8 

57.1 

7 

33.3 

42.9 

X17 . 

1 

12.5 

1 

14.2 

13.3 

X18 . 

3 

37.5 

0 

00.0 

20.0 

X19 . 

3 

37.5 

1 

14.2 

26.7 

X20 . 

3 

37.5 

5 

71.4 

53.3 

X21 . 

4 

50.0 

5 

71.4 

60.0 

X22 . 

3 

37.5 

0 

00.0 

20.0 

X23 . 

5 

62.5 

6 

85.7 

73.3 

X24 . 

2 

25.0 

5 

71.4 

46.7 

X25 . 

5 

62.5 

6 

85.7 

73.3 

X26 . 

6 

75.0 

7 

100.0 

86.7 

X27 . 

2 

25.0 

3 

42.8 

33.3 

X28 . 

1 

12.5 

3 

42.8 

26.7 

X29 . 

1 

12.5 

6 

85.7 

46.7 

X30 . 

4 

62.5 

5 

71.4 

60.0 

X34 . 

2 

25.0 

2 

28.5 

26.7 

X76 . 

0 

00.0 

1 

4.7 

2.8 

X77 . 

9 

64.2 

5 

23.8 

40.0 

X78 . 

4 

28.5 

11 

52.3 

42.8 

X79 . 

3 

21.4 

8 

38.0 

31.4 

X80 . 

8 

57.1 

6 

28.5 

40.0 

X81 . 

3 

60.0 

4 

44.4 

50.0 

X86 . 

5 

35.7 

9 

42.8 

40.0 

*  n=14  for  all  independent  variables  except,  X17-X34 

where  n=8 ,  and,  X81  where  n=5 

**  n®21  for  all  independent  variables  except,  X17-X34 

where  n=7,  and,  X81  where  n=9 

n=35  for  all  independent  variables  except,  X17-X34 
where  n=15,  and,  X81  where  n=14 
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responses  were  separated  and  analyzed  for  the  non-adequate 
manuals.  This  table  identified  the  following  possibilities. 

1.  The  majority  of  the  adequate  responses  were  from 
programs  in  the  deployment  phase  (42.8%).  whereas,  the 
majority  of  the  non-adequate  responses  were  in  the  full- 
scale  development  phase  (23.8%).  This  indicates  that 
changes  to  the  commercial  manuals  that  will  elevate  them  to 
an  adequate  level  occurs  between  the  full  scale  development 
phase  and  the  deployment  phase. 

2.  The  most  successful  programs  to  use  commercial 
manuals  are  Support  Equipment  programs  (35.7%);  whereas, 
commercial  manuals  are  least  successful  for  Space  Systems 
(47.6%)  . 

3.  The  reviews  that  were  accomplished  prior  to 
contract  award  benefited  the  final  manual.  57.1%  of  the 
fourteen  "yes”  responses  accomplished  pre-contract  reviews. 

4.  The  technical  order  specialist  (62.5%)  and  the 
using  command  representative  (62.5%)  were  the  key  reviewers 
for  the  pre-contract  reviews. 

5.  The  maintenance  procedures  (86.7%)  and  the  hardware 
accuracy  (60.0%)  were  the  key  elements  reviewed  in  both 
scenarios  (adequate  or  non-adequate);  however,  only  the  non- 
adequate  scenario  concentrated  on  a  thorough  review  for  data 
rights  restrictions,  part  number  accuracy,  and  safety 
warning  notices. 


6.  The  Data  Items  Description  (DID)  (64.2%)  and  the 
Statement  of  Work  (SOW)  (57.1%)  were  the  most  effective 
contract  vehicle  to  use  for  commercial  manuals;  while,  the 
CFE/CFAE  notices  (28.5%)  were  the  most  ineffective.  TMCR 
86-01  was  not  considered  a  complete  failure  due  to  its' 
limited  usage  to  date. 

7-.  Both  scenarios  found  errors  in  the  context  of  the 
manuals  and  required  some  additional  funding  (50.0%). 

Possibilities  4,  5,  and  7  were  then  further  analyzed 
for  validity.  The  other  possibilities  were  left  as  optional 
and  can  be  used  for  further  study  later. 

Table  III  compares  the  reviewer  to  the  review  element. 
Within  the  table,  the  figures  appear  as  a  fraction.  The  top 
figure  represents  the  total  number  of  the  reviewer (s)  from 
the  adequate  manuals  that  were  reviewed  for  the  specific 
elements.  Whereas,  the  bottom  figure  is  the  same  for  the 
non-adequate  commercial  manuals.  In  addition,  there  is  a 
line  drawn  after  the  first  four  elements.  Above  the  line 
are  the  reviewers  (X17,  X18,  X19,  and  X22)  that  participated 
in  the  reviews  for  adequate,  but,  rarely  participated  for 
non-adequate.  Below  the  line  are  the  reviewers  (X20,  X21, 
X23,  X24,  and  X25)  that  had  the  strongest  participation  in 
the  non-adequate  reviews. 

As  the  table  indicates,  the  first  four  reviewers 
checked  partially  for  the  maintenance  procedures  and  some 
for  the  hardware  accuracy.  They  did  not  check  data 
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restriction  issues,  part  number  accuracy  or  safety  notices. 
Even  though  the  review  teams  did  not  consist  of  only  these 
reviewers,  the  other  reviewers  basically  reviewed  for  the 
same  components.  Only  in  one  instance  did  the  other 
reviewers  check  for  the  data  restrictions,  part  number 
accuracy,  or  safety  notices. 


TABLE  III 


REVIEWER  VERSUS  REVIEWED 

Independent 
Variable  w/ 


Full  Potential 

X26 

X27 

X28 

X29 

X30 

X17 

(1/1) * 

1/1 

0/1 

0/1 

0/1 

0/1 

XI 8 

(3/0) 

2/0 

0/0 

0/0 

0/0 

1/0 

XI 9 

(3/1) 

2/1 

1/1 

0/0 

0/1 

1/0 

X22 

(3/0) 

2/0 

1/0 

0/0 

0/0 

1/0 

X23 

(5/6) 

3/6 

1/3 

1/3 

1/5 

2/4 

X21 

(4/5) 

3/5 

0/3 

0/3 

0/5 

0/4 

X20 

(3/5) 

1/5 

0/2 

0/2 

0/4 

2/3 

X24 

(2/5) 

1/4 

1/3 

1/2 

1/5 

1/4 

X25 

(5/6) 

4/6 

1/2 

1/3 

1/5 

2/5 

TOTAL  (29/29) 

19/28 

5/15 

3/14 

3/26 

10/21 

%  adequate 

65.5 

17.2 

10.3 

10.3 

34.5 

%  non-adequate 

96.5 

51.7 

48.3 

90.0 

72.4 

*NOTE;  Adequate/Non-adequate 


On  the  other  hand,  the  non-adequate  review  teams  were 
comprised  mostly  of  the  technical  order  specialist,  the 
equipment  specialist,  the  logistics  management  specialist, 
the  maintenance  technician,  and  the  using  command.  The 
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main  emphases  of  their  review  were  on  the  maintenance 
procedures,  the  safety  notices,  and  the  hardware  accuracy. 
Part  number  accuracy  and  data  restrictions  were  a  concern, 
but  only  to  a  lesser  degree. 

This  table  svimmarizes  that  the  manuals  deemed  non- 
adequate  were  reviewed  for  all  of  the  review  elements. 

While,  the  adequate  manuals  lacked  a  thorough  review.  It 
can  be  assumed  that  the  only  reason  for  the  hardware 
accuracy  not  being  100%  for  the  non-adequate  reviews  was  due 
to  the  lack  of  equipment  availability. 

Table  IV  summarizes  the  number  of  errors  found  during 
the  pre-contract  review  to  the  type  of  review  and  whether  or 
not  additional  funding  was  required.  Again,  the  table 
compares  the  adequate  responses  to  the  non-adequate 
responses.  From  the  table,  the  assumption  can  be  made  that 
the  majority  of  errors  were  due  to  faulty  maintenance 
procedures.  In  addition,  the  categories  with  the  majority 
of  maintenance  changes  also  required  extra  funding.  There 
is  the  possibility  that  the  extra  funding  was  due  to  lack  of 
part  number  accuracy,  safety  notices,  or  hardware  accuracy. 

The  final  table  compares  the  commercial  manual  usage  at 
the  depot  and  the  intermediate  level  to  the  programs  that 
will  also  use  interim  contract  support  (ICS)  and  additional 
funding.  Table  V  indicates  that  at  the  depot  level,  of  the 
fourteen  adequate  responses,  seven  will  utilize  ICS  at  the 
depot  and  six  at  the  intermediate  level.  Of  this,  two 
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TABLE  IV 


THE  NUMBER  OF  ERRORS  IN  COMPARISON  TO 
ADDITIONAL  FUNDING  REQUIREMENTS 


#  of  changes 
w/  Full  Potential 

X26 

X27 

X28 

X29 

X30 

X81 

0-1 

(2/1)  * 

2/1 

1/1 

0/0 

0/1 

1/0 

1/1 

3-5 

(0/2) 

0/2 

0/1 

0/1 

0/1 

0/1 

0/1 

6-8 

(0/0) 

0/0 

0/0 

0/0 

0/0 

0/0 

0/0 

9-11 

(0/0) 

0/0 

0/0 

0/0 

0/0 

0/0 

0/0 

11+ 

(0/3) 

0/3 

0/0 

0/1 

0/3 

0/3 

0/1 

TOTAL 

(2/6) 

2/6 

1/2 

0/2 

0/5 

1/4 

1/3 

% 

100/100 

50/33 

0/33 

0/33 

50/66 

*NOTE:  Adequate/Non-adequate 


TABLE  V 

INTERIM  CONTRACT  SUPPORT/ADDITIONAL  FUNDING  REQUIREMENTS 


Interim 

Contract 

Support 

Required 

% 

Additional 

Funding 

Required 

% 

Depot 

7/13 

59/62 

2/1 

28.5/7. 

Adequate 

6/6 

43/28.5 

3/2 

50/33 

Note:  The  percentages  are  based  upon  n=14  (adequate)  and 

n=21  (non-adequate) . 
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required  additional  funding.  For  the  twenty-one  non- 
adequate  responses,  thirteen  required  ICS  at  the  depot  level 
and  six  at  the  intermediate  level.  Of  the  non-adequate ,  one 
of  the  depot  and  two  of  the  intermediate  required  additional 
funding.  These  figures  indicate  that  up  to  50%  of  the 
programs  that  utilize  commercial  manuals  requires  ICS  and  up 
to  50%  of  these  program  require  additional  funding. 

Overall  Statistical  Findings 

In  summary,  the  comparison  analysis  indicates  that 
commercial  manuals  are  not  adequate  for  use  by  5  and  7  level 
maintenance  technicians  unless  supplements  are  provided  and, 
in  some  instances,  interim  contract  support  is  given.  Both 
additions  can  add  thousands  of  dollars  each  year  to  the 
operations  and  support  budget.  For  example,  observation  50 
of  the  survey  required  an  additional  $587K  to  upgrade  the 
commercial  manuals  in  order  to  bring  them  to  an  adequate 
standard.  However,  on  the  reverse  side,  observation  58 
required  additional  funding  to  produce  MIL-SPEC  technical 
orders  when  the  commercial  manuals  could  not  be  supplemented 
adequately . 

If  only  40%  of  the  commercial  manuals  were  considered 
adequate,  the  question  must  be  asked,  "were  they  really 
adequate?"  The  survey  results  indicated  that  66.7%  were 
reviewed  for  maintenance  procedures,  13.5%  for  data 
restrictions,  11.1%  for  part  number  accuracy  and  safety 
notices,  and  37.0%  for  hardware  accuracy. 
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Interview  Results 


To  obtain  a  knowledge  of  the  commercial  vendors 
procedures,  an  informal  interview  was  accomplished  with 
General  Electric  (GE) . 

The  GE  interview  was  held  with  Hazel  Hackney,  Director 
of  Maintenance  Data  Operations,  and  John  Bowers,  Technical 
Publications  Manager.  Each  representative  was  asked  to 
openly  discuss  the  contractors  procedures  in  commercial 
equipment  buys  by  non-military  industries,  and  how  they 
differ  from  military  contracts. 

Two  major  differences  were  found.  As  Ms.  Hackney 
stated,  "it's  easier  to  operate  in  the  commercial  world  than 
to  adhere  to  military  specification." 

The  first  major  difference  is  the  vehicle  used  on  a 
contract  to  obtain  the  commercial  manuals.  In  the  military 
there  are  at  least  five  vehicles:  Contractor  Furnished 
Equipment/Contractor  Furnished  Aeronautical  Equipment 
Notices,  Air  Force  Acquisition  Directives,  Statements  Of 
Work,  Technical  Manual  Contract  Requirement  86-01,  and  Data 
Items  Descriptions.  In  the  commercial  world,  the  contract 
is  let  with  one  standard  specification  that  is  used 
worldwide.  This  specification  is  Air  Transport  Association 
(ATA)  -100  and  -200.  The  100  series  establish  the  format 
(full  page) ,  the  numbering  system,  and  the  chapters  for  the 
manuals  and  service  bulletins.  While  the  200  series 
establish  the  same  for  the  illustrated  parts  catalogs  and 
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provisioning.  (Currently,  the  ATA-200  is  under  revision  and 
will  be  republished  as  ATA~2000.) 

There  are  three  reasons  for  the  universal  support  of 
the  ATA  specifications.  They  are: 

1.  All  specifications  are  updated  or  revised  at  least 
once  a  year.  An  annual  meeting  is  held  for  representatives 
of  the  airline  industry  to  present  change  proposals.  Prior 
to  the  meeting's  completion,  the  changes  are  either  approved 
or  disapproved  and  implemented  into  the  specification  as  an 
update . 

2.  All  industries  (electronics,  communications,  etc.) 
abide  by  these  specifications.  When  a  company  sees  this 
specification  on  contract,  they  are  already  familiar  with 
the  publication  requirements. 

3.  These  specifications  are  used  worldwide.  They  are 
not  limited  to  use  in  the  United  States  only. 

Also,  the  ATA  series  specifications  are  used  to 
establish  a  universal  data  base  that  will  be  used  by  each 
company.  This  data  base  allows  speedy  corrections  and  easy 
accessibility  to  any  manual,  generic  or  specialized.  Within 
this  data  base,  the  commercial  manuals  are  filed  by  task 
numbers.  The  first  three  numbers  identify  the  part  of  the 
aircraft  (engine) ,  the  module  the  part  is  located  in,  and 
the  piece  part  of  the  engine.  The  task  number  then 
continues  and  tells  the  function  (repair  or  disassembly) , 
and  the  type  of  repair  (weld,  etc.).  As  a  result,  the  sub- 


vendors  of  an  end  item  then  set  up  a  Reliability  and 
Maintainability  (R&M)  data  base  using  the  same  task  numbers. 
The  R&M  data  base  is  used  to  identify  the  level  of  repair, 
whether  the  item  is  a  consumable,  the  task  level,  etc.  (The 
data  collected  in  this  data  base  is  similar  to  the 
information  recorded  on  the  D  sheet  of  the  Logistics  Support 
Analysis  Records.)  The  sub-vendors  submit  the  R&M 
information  to  the  prime  contractor  on  magnetic  tape. 
Currently,  the  ATA  is  working  on  a  procedure  where  all  of 
the  information  will  be  passed  on  in  the  form  of  compact- 
disk  (CD)  . 

The  second  major  difference  between  the  non-military 
contracts  and  the  military  contracts  is  publication  changes 
or  supplements.  In  the  non-military  environment,  the 
rewriting  is  done  by  a  technical  writing  department,  either 
in  or  out  of  house  at  the  customers  facility. 

When  the  commercial  manual  is  procured  through  the  ATA 
specification,  the  customer  is  given  a  two  year 
"verification"  period  to  find  procedural  errors.  The 
specification  does  not  provide  for  an  official  verification 
against  the  manuals  or  the  changes,  however,  the  company 
"feels"  confident  that  the  manuals  have  been  reviewed  by 
competent  individuals  and  are  accurate  to  the  best  extent 
possible.  If  procedural  errors  are  submitted  back  to  the 
original  contractor  in  the  form  of  a  Publication 
Clarification  Request  (the  contractor's  equivalent  to  the 
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AFTO-22)  at  no  expense  to  the  customer  (unless  the 

contractor  deems  the  request  as  non-procedural.)  Once  | 

received,  the  technical  publications  department  reviews  the 
submittal  and  responds:  1)  Good  idea,  further  research;  2) 

Bad  idea,  no  further  research;  or,  3)  Bad  idea,  fund  and  we  j 

will  research.  These  submittals  are  accumulated  for  90 
days.  At  the  end  of  every  90  days,  the  publication 

clarification  request  are  put  into  a  change  notice  and  | 

distributed  to  all  manual  holders.  Changes  required  in 
formatting  or  rewording,  the  customer  has  bought  the 

commercial  manuals  "as-is".  It  is  left  up  to  the  customers  j 

technical  writers  to  rewrite.  Legal  implications  are 
avoided  because  the  prime  contractor  has  the  original  in  a 
data  base  and  does  not  assume  responsibility  for  the  changes 
made  by  the  customer  after  purchase. 


I 


I 


I 


I 
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IV.  Conclusions  and  Recommendations 


I 

Significant  Results  of  Survey 

The  statistical  findings  support  the  first  part  of  the 
hypothesis.  The  hypothesis  states  that  the  cost 
effectiveness  of  commercial  manuals  is  not  always  present 
when  tested  against  adequacy,  reverse  engineering,  or 
maintenance  turnaround  time.  Due  to  improper  survey 
questions,  reverse  engineering  and  maintenance  turnaround 
time  were  not  tested  effectively.  However,  the  analysis  did 
indicate  four  significant  facts.  They  are: 

1.  The  review  process  is  currently  inadequate. 

2.  The  DID  and  the  SOW  are  the  most  effective  contract 
medium  for  adequate  commercial  manuals. 

3.  Early  involvement  in  the  acquisition  program  phase 
is  important  when  reviewing  the  commercial  manuals. 

4.  Additional  expense  is  usually  incurred  prior  to  the 
final  acceptance  of  the  manual  by  the  Air  Force. 

The  survey  did  support  the  existence  of  inadequacy  in 
commercial  manuals.  At  first,  the  overall  percentage  of 
40.0%  adequate  and  60.0%  non-adequate  leads  one  to  believe 
that  the  manuals  have  questionable  adequacy.  However,  when 
a  closer  look  was  taken  at  what  caused  the  commercial 
manuals  to  be  adequate,  it  was  found  that  the  adequate 
manuals  were  not  reviewed  thoroughly  for  maintenance 
procedures,  safety  and  warning  notices,  data  rights,  part 
number  adequacy,  or  hardware  adequacy.  Also,  the  adequate 


manuals  were  reviewed  by  personnel  having  the  least 
knowledge  on  the  day  to  day  functions  of  technical  manual 
development.  Whereas,  the  manuals  found  to  be  inadequate 
were  reviewed  by  personnel  intimate  with  the  process,  i.e. 
the  technical  order  specialist,  the  equipment  specialist, 
the  logistics  management  specialist,  the  maintenance 
technician,  and  the  using  command.  In  addition,  they  were 
reviewed  for  the  maintenance  procedures,  safety  procedures, 
hardware  accuracy,  data  rights,  and  part  numbers.  (This  has 
been  listed  in  the  order  of  precedence  calculated  through 
the  survey  instrument.) 

The  second  significant  finding  indicates  that  the  full 
scale  development  phase  is  crucial.  The  survey  indicated 
the  highest  percentage  (23.8%)  of  the  manuals  were  non- 
adequate  at  full  scale  development,  while,  the  highest 
percentage  (42.8%)  of  the  manuals  were  adequate  during  the 
deployment  phase.  Therefore,  it  is  assumed  that  the  system 
has  been  developed  enough  to  test  the  manuals.  Also,  it  is 
early  enough  to  begin  supplementing  if  necessary  and  be 
ready  for  deployment. 

The  survey  also  support  that  the  additional  cost  of 
adequate  commercial  manuals  is  not  realized  by  the  Air  Force 
until  a  system  has  reached  the  production  or  deployment 
phase.  This  additional  cost  of  using  commercial  manuals  is 
associated  with  interim  contractor  support  and  technical 
order  supplements. 
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Of  the  thirty  five  observations,  fourteen  were  in  the 
deployment  phase  and  eight  were  in  the  production  phase.  Of 
the  fourteen,  only  five  systems  had  adequate  commercial 
manuals  and  nine  used  interim  contract  support  (ICS)  up  to 
six  years.  Also,  of  the  eight  only  two  were  adequate  with 
four  using  ICS  from  two  to  over  ten  years.  This  averages 
out  to  only  31.8%  of  the  overall  commercial  manuals  used  by 
the  maintainers  were  adequate. 

In  addition,  three  of  the  fourteen  observations 
required  additional  funding  for  technical  manual  changes  of 
25.5%  or  over.  Only  in  one  instance  was  it  for  a  change 
that  involved  only  2.5%  of  all  of  the  manuals.  As  for  the 
eight  production  phase  observations,  two  required  additional 
funding  for  changes  that  encompassed  over  15.5%  of  the 
manuals.  The  survey  did  not  examine  whether  or  not  these 
changes  involved  operational  or  format  procedures. 

In  eight  out  of  the  fourteen  observations,  during  the 
full  scale  development  phase,  the  contractor  was  required  by 
contract  to  make  changes  and  updates.  However,  in  two  of 
the  eight,  additional  funding  was  required.  One  of  the  two 
was  for  a  change  involving  25.5%  of  the  manual.  In  five  of 
the  eight  observations,  the  contractor  was  required  by 
contract  to  make  changes  and  updates.  However,  in  one  of 
the  five,  where  more  that  25.5%  of  the  manual  was  changed, 
additional  funding  was  required. 


The  Packard  Commission's  31ue  kibbon  Study  indicated 
that  the  use  of  the  commercial  equipment  was  cost  effective 
since  the  DoD  did  not  fund  the  research  and  development.  In 


addition,  the  commercial  equipment  could  be  used  with 
relatively  few  changes  to  the  hardware  or  the  commercial 
manuals.  However,  this  study  indicates  that  procedural 
changes  in  the  review  process  must  be  accomplished  in  order 
to  achieve  this  "cost  effectiveness."  Initially,  the 
manuals  should  be  reviewed  thoroughly  by  a  standardized 
review  process.  Second,  the  DoD  should  implement  procedures 
similar  to  the  commercial  market  when  putting  commercial 
manuals  on  contract. 

First,  this  study  must  address  what  is  an  adequate 
manual.  For  the  purposes  of  the  results,  an  adequate  manual 
must  be  usable  for  operations  and  maintenance  to  include 
correct  maintenance  procedures,  accurate  part  numbers, 
accurate  hardware  descriptions,  pertinent  safety  or  warning 
notices,  and  limited  data  restrictions. 

As  the  statistical  findings  actually  indicate,  the 
"adequate"  manuals  are  not  always  adequate.  Steps  must  be 
taken  to  assure  the  manuals  are  corrected  prior  to  the 
deployment  phase  with  minimal  additional  cost.  The  manuals 
should  be  reviewed  as  early  as  possible,  similar  to  the 
review  process  of  the  MIL-SPEC  manuals.  As  soon  as  a  piece 
of  commercial  equipment  is  identified  for  use  on  a  system. 


the  manuals  should  be  reviewed  for  compliance  with  MIL-M- 
7298C  by  the  technical  order  specialist,  the  equipment 
specialist,  the  logistics  management  specialist,  and  the 
using  command.  The  number  of  days  allowed  for  the  review 
should  be  decided  by  the  number  of  manuals  to  be  reviewed  in 
one  period  and  not  by  a  standard  30  day  review  period.  For 
example,  if  the  contractor  submits  twenty  manuals  to  be 
reviewed,  it  should  not  be  expected  of  the  government  to 
have  them  thoroughly  reviewed  in  thirty  calendar  days.  The 
contract  should  allow  two  days  per  manual.  (Two  days  was 
selected,  because  a  small  manual  (30-60  pages)  can  be 
reviewed  in  approximately  one  day,  whereas  a  large  manual 
(over  200  pages)  can  be  reviewed  in  approximately  three 
days.  This  averages  to  two  days.) 

Once  the  system  has  been  integrated,  the  .manuals  should 
be  verified  along  with  the  MIL-SPEC  manuals.  If  technical 
errors  are  found,  and  the  error  is  not  caused  by  the  system 
integration  or  personal  preference  (i.e.  format),  then  the 
contractor  should  be  responsible  for  the  change  at  no 
additional  cost  to  the  government. 

Also,  as  errors  are  found  and  changes  are  made  by  other 
commercial  customers,  the  change  or  supplement  should  be 
forwarded  to  the  government. 

If  the  DoD  continues  with  this  philosophy,  they  should 
consider  developing  a  universal  specification  that  is  agreed 
upon  by  all  departments  (Air  Force,  Army,  Navy,  Marines). 
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This  specification  should  be  modeled  after  the  ATA  series 
used  by  commercial  customers  of  the  industry.  Some  examples 
of  what  the  specification  would  cover  would  include  format, 
changes,  and  numbers.  For  format,  the  requirement  would 
mirror  the  contract  requirement.  If  we  are  going  to  use 
commercial  equipment  to  reduce  cost,  then  we  should  use  the 
commercial  manuals  in  the  same  manner.  If  changes  were 
approved  as  an  operational  change  and  not  just  a  format 
change,  the  contractor  would  be  responsible.  All  other 
changes  would  be  the  government's  responsibility.  For 
number  assignment,  a  cover  sheet  would  identify  the  DoD  j 

department  number  with  a  note  referring  back  to  the 
contractors  number  for  the  manual.  This  specification  would 
require  an  elite  group  to  gather  and  research  possible  j 

changes  to  the  specification  continually.  Annually,  the 
group  would  meet  with  the  commercial  ATA  representatives  and 
review  the  accumulation  of  DoD  approved  recommendations.  | 

Once  the  commercial  side  also  approved  the  changes,  they 
would  be  incorporated  into  a  revision  to  the  original 

specification.  (Revisions  released  after  the  date  of  | 

contract  award  would  not  effect  the  current  contract.) 

As  far  as  putting  the  specification  on  contract,  the 
specification  would  have  its'  own  Data  Item  Description  | 

(DID) .  Tailoring  or  the  DID  could  be  accomplished  in  the 
contract  data  requirements  list  and  the  statement  of  work. 

j 
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In  addition  to  developing  the  specification,  the  DcD 
should  consider  using  the  same  or  similar  hardware  as  the 
commercial  world.  The  commercial  world  is  beginning  to 
write  and  sell  the  commercial  manuals  on  compact  disk  (CD) . 
This  requires  a  CD  reader  that  attaches  to  the  personal 
computers.  The  CD  reader  would  allow  the  user  to  make  the 
required  changes.  As  previously  noted,  the  contractor  is 
not  liable  for  the  changes  made  by  the  customer. 

I  do  not  recommend  that  each  user  be  given  the 
opportunity  to  automatically  make  changes.  However,  I  do 
recommend  that  the  DoD  consider  instituting  a  specialized 
technical  writing  department  that  will  make  appropriate 
changes  and  distributions.  This  would  reduce  the 
contractors  cost  of  setting  up  individual  data  bases  for  the 
military  and  the  supplementing  cost  for  integrated  and 
militarized  commercial  equipment. 

Recommendations  for  Follow-on  Studies 

There  are  three  studies  that  should  be  accomplished.  The 
first  would  analyze  the  cost  of  setting  up  a  specialized 
department  for  writing  commercial  manual  changes  internally 
versus  contracting  to  civilian  technical  writing  firms.  The 
second  would  expand  the  questionnaire  to  include  whether  or 
not  the  MIL-SPEC  manuals  reviewed  in  section  three  were 
adequate.  If  the  majority  were,  then  appropriate  reviews 
should  be  required  of  the  commercial  manuals  in  order  to 
find  out  operational  errors  and  to  negotiate  changes  to  the 


manuals  while  early  in  the  acquisition  cycle.  The  third 
study  would  expand  the  questionnaire  to  analyze  the 
successfulness  of  the  reviews  under  MIL-M-7298C. 

Finally,  the  latter  two  studies  could  be  compared  and 
the  "specifics"  of  a  commercial  manual  review  could  be 
established . 
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Appendix  A: 


Questionnaire  and  Response  Totals 


Purpose :  The  objective  of  the  research  is  to  deternine 

whether  or  not  couunercial  manuals  are  the  most  cost 
effective  method  when  considering  the  cost  of  buying 
adequate  documentation  versus  the  cost  of  developing  MIL- 
SPEC  manuals,  the  cost  of  reverse  engineering,  and  the 
increase  in  maintenance  turnaround.  This  information  will 
aid  in  the  analysis  of  an  important  Air  Force  problem. 

Definitions 


Commercial  Manuals:  Commercial  publications  contain 
technical  information  on  the  assembly,  installation, 
service,  disassembly,  overhaul  and  assembly  of  equipment 
and  parts  identification.  These  publications  are  manuals, 
booklets,  or  like  data  that  are  furnished  by  manufacturers 
to  purchasers  of  their  products.  [TO  00-5-01,  2-7] 

Commercial  Off  the  Shelf  Equipment:  An  item  developed  and 
used  for  other  than  government  purposes  sold  or  traded  to 
the  general  public  in  the  course  of  normal  business 
operations  and  used  unchanged  when  acquired  by  the 
government.  [FAR  11.001] 

Custom  Products:  A  commercial  item  modified  to  meet  some 
government-peculiar  physical  requirement  or  addition  or 
otherwise  identified  differently  from  its  normal  commercial 
counter  parts.  [FAR  11.001] 

Standardization:  The  process  by  which  member  nations 

achieve  the  closest  practicable  cooperation  among  forces, 
the  most  efficient  use  of  research  development  and 
production  resources,  and  agreement  to  adopt  on  the  broadest 
possible  basis  the  use  of:  (1)  common  or  compatible 
operational,  administrative,  and  logistics  procedures;  (2) 
common  or  compatible  technical  procedures  and  criteria:  (3) 
common,  compatible,  or  interchangeable  supplies,  components 
weapons,  or  equipment;  and  (4)  common  or  compatible  tactical 
doctrine  with  corresponding  organizational  compatibility. 
[AFR  800-34] 

Direction:  Please  specify  the  answer  to  the  corresponding 

number  {A=l,  B=2,  C=3,  D=4,  E=5)  on  the  attached  opti-scan 
form  using  a  No.  2  lead  pencil.  For  questions  stating 
"other,  please  specify,"  attach  a  separate  sheet  with  the 
corresponding  question.  If  a  question  is  not  applicable  to 
the  selected  end  item,  leave  blank.  If  more  than  one  answer 
is  applicable,  check  all  of  the  correct  answers.  Thank  you. 

USAF  Survey  Control  No.  88-40 
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Section  One 


1  =  Program  Phase 


Conceptual  Phase  (1) 

1 

2.3% 

Demonstration/Validation  Phase  (2) 

0 

0.0% 

Full  Scale  Development  Phase  (3) 

11 

31.4% 

Production  Phase  (4) 

8 

22.8% 

Deployment  Phase  (5) 

15 

42.8% 

2 

s 

Maintenance  Concept 

2-level  (Organization,  Depot)  (1) 

6 

17.1% 

3-level  (Organization,  Intermediate, 

Depot) 

(2) 

22 

52.3% 

Other  (0) 

7 

20.0% 

3 

= 

Is  the  depot  level  maintenance  done 

through 

organic 

support? 

Yes  (1) 

19 

54.3% 

No  -  (0) 

16 

45.7% 

4 

Is  the  depot  level  maintenance  done 

through 

contract 

support? 

Yes  (1) 

21 

60.0% 

No  (0) 

14 

40.0% 

5 

s 

Is  the  intermediate  maintenance  done 

through 

organic 

support? 

Yes  (1) 

13 

51.4% 

No  (0) 

17 

48 . 6% 

6 

s 

Is  the  intermediate  maintenance  done 

through 

contract 

support? 

Yes  (1) 

12 

34.3% 

No  (0) 

23 

65.7% 

7 

s 

How  many  years  will  interim  contract 

support 

be 

used? 

Less  than  1  year  (.5) 

5 

14.3% 

1-3  years  (2) 

16 

45.7% 

4-6  years  (5) 

9 

25.7% 

7-10  years  (8.5) 

1 

2.8% 

Over  10  years  (11) 

1 

2.8% 

No  response 

3 

3.6% 

8 

s 

End  item  quantity  of  the  system 

1-5  (3) 

5 

14.3% 

6-15  (10.5) 

7 

20.0% 

16-30  (23) 

3 

8.6% 

31-50  (40.5) 

4 

11.4% 

Over  50  (51) 

16 

45.7% 
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9  =  Equipment  type 

Communications  Equipment  (1) 
Support  Equipment  (2) 
Aircraft  (3) 

Computer  Resources  (4) 

Space  System  (5) 

Other  (0) 


5  14.3% 
10  28.6% 
9  25.7% 
2  5.7% 
5  14.3% 
4  11.4% 


10  =  Was  a  cost  comparison  done  between  the  commercial 

manual  and  supplemental  cost  versus  MIL-STD  technical 
order  development  cost? 

Yes  (1)  23  65.7% 

No  (0)  12  34.3% 


Section  Two 


11  =  Are  the  commercial  manuals  used  at  the  intermediate 
facility? 

Yes  (1)  22  52.3% 

No  (0)  13  37.1% 


12  =  Are  the  commercial  manuals  used  at  the  depot  facility? 
Yes  (1)  23  65.7% 

No  (0)  12  34.3% 


13  =  Commercial  manuals  make-up  what  percent  of  the  total 
system  manuals? 

Less  then  10%  (.05)  8 

10-30%  (.2)  7 

31-50%  (.405)  5 

Over  50%  (.51)  12 

No  response  2 


22.8% 

20.0% 

14.3% 

34.3% 

5.7% 


14  =  Did  the  contract  require  the  contractor  to  review  the 
commercial  manuals  in  accordance  with  MIL-M-7298C? 

Yes  (1)  22  62.8% 

No  (0)  13  37.1% 


15  =  Was  the  contractor  or  the  government  required  to 

review  the  commercial  manuals  prior  to  contract  award? 
Yes  (1)  15  42.8% 
No  (0)  20  57.1% 
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*Variables  16  ~  43  were  only  answered  if  variable  15  was 
yes . 


16  =  Nuznber  of  days  allotted  to  review  the  manuals  prior 


contract  award. 

0-10  days  (5)  4  26 
11-30  days  (20.5)  4  26 
31-60  days  (45.5)  6  40 
Over  60  days  (61)  0  0 
No  Response  1  6 


Government  representatives  that  reviewed  the  commer 
manuals 


17  =  Integrating  Contractor 

Yes  (1)  2  13 

No  (0)  13  86 


13  =  Logistics  Support  Contractor  • 

Yes  (1)  3  20 

No  (0)  12  30 


19  =  Program  Manager 
Yes  (1) 

No  (1) 


4  26 

11  73 


20  =  Logistics  Manager 
Yes  (1) 

No  (0) 


7  46 

8  53 


21  =  Equipment  Specialist 
Yes  (1) 

No  ( 0 ) 


8  53 

7  46 


Engineer 
Yes  (1) 
No  (0) 


3  20 

12  80 


23  =  Technical  Order  Specialist 
Yes  (1) 

No  (0) 


10  66 
5  33 


24  =  Maintenance  Technician 

Yes  (1)  7  46 

No  (0)  8  53 


25  =  Using  Command 

Yes  (1)  10  66 

No  (0)  5  33 


to 

.  4% 

.  4% 

.  0% 
.0% 
.7% 

cial 


.  3% 
.7% 

.  0% 
.  0% 

.1% 

.3% 


•  i  0 

.  3% 


.  3% 
.7% 


.  0% 
.  0% 


.7% 
.  3% 


.7% 
.  3% 


.7% 
.  3% 
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Purpose  of  the  commercial  manual  review 

26  =  Maintenance  Procedures 

Yes  (1) 

No  (0) 

27  =  Data  rights  Restrictions 

Yes  (1) 

No  (0) 

28  =  Part  Number  Accuracy 

Yes  (1) 

No  (0) 

29  =  Safety  Naming  Notices 

Yes  (1) 

No  (0) 

30  =  Hardware  Accuracy 

Yes  (1) 

No  (0) 


12  30.0% 

3  20.0% 


5  33.3% 

10  66.7% 


3  20.0% 

12  80.0% 


5  40.0% 

9  60.0% 


8  53.3% 

7  46.7% 


Type  of  quality  check  used  for  the  commercial  manuals 


31  =  Validation 
Yes  (1) 

No  (0) 


5  33.3% 

10  66.7% 


32  =  Verification 
Yes  (1) 

No  (0) 


6  40.0% 

9  60.0% 


33  =  In  Process  Review 
Yes  (1) 

No  (0) 


4  26.7% 

11  73.3% 


34  =  If  a  verification,  validation,  or  in-process  review 
was  accomplished,  were  there  differences  between  the 
original  operational  procedures  and  the  actual 
findings? 

Yes  (2)  2  13.3% 

No  (1)  3  20.0% 

Do  not  know  (0)  10  66.7% 

Location  of  the  commercial  manual  review 


35  =  Contractor's  Facility 

Yes  (1)  7  46.7% 

No  (0)  8  53.3% 
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36  =  Air  Force  Facility 

Yes  (1)  5  40.0% 

No  (0)  9  60.0% 

37  =  Was  the  actual  equipment  available  during  the 

commercial  manual  review? 

Yes  (1)  6  40.0% 

No  (0)  9  60.0% 

33  =  Was  the  equipment  used  during  the  commercial  manual 
review? 

Yes  (1)  6  40.0% 

No  (0)  9  60.0% 

39  =  Reading  grade  level  of  the  commercial  manuals 

Ninth  grade  (1)  3  20.0% 

Twelfth  grade  (2)  0  00.0% 

Fourteenth  grade  (3)  2  13.3% 

Other  (4)  1  6.7% 

No  requirement  (0)  9  60.0% 

40  =  Average  number  of  volumes  per  commercial  manual 

1  (1)  5  33.3% 

2  (2)  2  13.3% 

3  (3)  1  6.7% 

4  (4)  3  20.0% 

5  or  more  (5)  3  20.0% 

No  Response  1  6.7% 

41  =  Are  detailed  diagrams  or  graphics  used  to  show  the 

maintenance  procedures  step  by  step? 

Yes  (1)  6  40.0% 

No  (0)  9  60.0% 

42  =  Does  an  illustrated  parts  breakdown  come  with  the 

commercial  manuals? 

Yes  (1)  11  73.3% 

No  (0)  4  26.7% 

43  =  Do  the  illustrated  parts  breakdown  diagrams  cross 

reference  each  other  and  the  text? 

Yes  (1)  5  33.3% 

No  (0)  10  66.7% 

Section  Three 

44  =  Are  the  MIL-SPEC  manuals  used  at  the  intermediate 

facility? 

Yes  (1)  24  68.6% 

No  (0)  11  31.4% 
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45 

= 

Are  the  MIL-SPEC  manuals 

used  at  the  depot  facility? 

Yes  (1) 

21 

60.0% 

No  (0) 

14 

40.0% 

46 

MIL-SPEC  manuals  make-up 

what  percent  of  the  total 

system  manuals? 

Less  then  10%  (.05) 

4 

11.4% 

10-30%  (.20) 

3 

8.6% 

31-50%  (.405) 

4 

11.4% 

Over  50%  (.51) 

20 

57.1% 

No  response 

4 

11.4% 

47 

= 

Did  the  contract  require 

the  contractor  to  review  the 

MIL-SPEC  manuals  in  accordance  with  MIL-M-38734? 

Yes  (1) 

23 

65.7% 

No  (0) 

12 

34.3% 

Type  of  quality  check  used 

for  the  MIL-SPEC  manuals 

43 

= 

Validation 

Yes  (1) 

25 

71.4% 

No  (0) 

9 

25.7% 

No  response 

1 

2.3% 

49 

= 

Verification 

Yes  (1) 

22 

62.8% 

No  (0) 

13 

37.1% 

50 

= 

In  Process  Review 

Yes  (1) 

2 

5.7% 

No  (0) 

33 

94.3% 

Government  representatives 

that  reviewed  the  MIL- 

-SPEC 

manuals 

51 

Integrating  Contractor 
Yes  (1) 

14 

40.0% 

No  (0) 

21 

60.0% 

52 

= 

Logistics  Support  Contractor 

Yes  (1) 

9 

25.7% 

No  (0) 

26 

74.3% 

53 

= 

Program  Manager 

Yes  (1) 

8 

22.8% 

No  (0) 

27 

77.1% 

54 

= 

Logistics  Manager 

Yes  (1) 

13 

37.1% 

No  (0) 

22 

62.8% 
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55  =  Equipment  Specialist 


Yes  (1) 

21 

60.0% 

No  (0) 

14 

40.0% 

56 

= 

Engineer 

Yes  (1) 

12 

34.3% 

No  (0) 

23 

65.7% 

57 

= 

Technical  Order  Specialist 

Yes  (1) 

22 

62.8% 

No  (0) 

13 

37.1% 

58 

= 

Maintenance  Technician 

Yes  (1) 

19 

54.3% 

No  (0) 

16 

45.7% 

59 

= 

Using  Command 

Yes  (1) 

24 

63.6% 

No  (0) 

11 

31.4% 

Purpose  of  the  MIL-SPEC  manual  review 

60 

= 

Maintenance  Procedures 

Yes  (1) 

24 

68.6% 

No  (0) 

11 

31.4% 

61 

= 

Data  rights  Restrictions 

Yes  (1) 

7 

20.0% 

No  (0) 

28 

80.0% 

62 

= 

Part  Number  Accuracy 

Yes  (1) 

20 

57 . 1% 

No  (0) 

15 

42.8% 

63 

Safety  Warning  Notices 

Yes  (1) 

18 

51.4% 

No  (0) 

17 

48.6% 

64 

= 

Hardware  Accuracy 

Yes  (1) 

22 

62.8% 

No  (0) 

13 

37.1% 

65 

= 

If  a  verification,  validation,  or  in-process  review 

was  accomplished,  were  there  differences 

between 

the 

original  operational  procedures  and  the 

actual 

findings? 

Yes  (2) 

18 

51.4% 

No  (1) 

5 

14.3% 

None  accomplished  (0) 

12 

34.3% 

61 


Location  of  the  MIL-SPEC  manual  review 


Contractor's  Facility 


Yes  (1) 

25 

71,4% 

No  (0) 

10 

28.6% 

Air  Force  Facility 

Yes  (1) 

14 

40.0% 

No  (0) 

21 

60.0% 

Was  the  actual  equipment  available  during  the 
commercial  manual  review? 

Yes  (1) 

17 

43.6% 

No  (0) 

18 

51.4% 

Was  the  equipment  used 

during  the  commercial  manual 

review? 

Yes  (1) 

13 

51.4% 

No  (0) 

17 

48.6% 

Number  of  days  allotted  to  review  the  manuals. 

0-10  days  (5) 

10 

28.6% 

11-30  days  (20.5) 

7 

20.0% 

31-60  days  (45,5) 

3 

8.6% 

Over  60  days  (61) 

6 

17.1% 

No  response 

9 

25.7% 

Reading  grade  level  of 

the  MIL-SPEC  manuals 

Ninth  grade  (1) 

6 

17.1% 

Twelfth  grade  (2) 

8 

22.3% 

Fourteenth  grade  (3) 

2 

5.7% 

Other  (4) 

0 

0.0% 

No  requirement  (0) 

19 

54.3% 

Average  number  of  volumes  per  MIL-SPEC  manual 

1  (1) 

9 

25.7% 

2  (2) 

4 

11.4% 

3  (3) 

4 

11.4% 

4  (4) 

1 

2.8% 

5  or  more  (5) 

8 

22.8% 

No  response 

9 

25.7% 

Are  detailed  diagrams 

or  graphics  used  to  show 

the 

maintenance  procedures 

step  by  step? 

Yes  (1) 

14 

40.0% 

No  (0) 

21 

60.0% 

Does  an  illustrated  parts  breakdown  come  with  the 
MIL-SPEC  manuals? 

Yes  (1)  19  54.3% 


75  =  Do  the  illustrated  parts  breakdown  diagrams  cross 
reference  each  other  and  the  text? 

Yes  (1)  18 

No  (0)  17 

Section  Four 

Commercial  manual  contract  requirements 


76  =  TMCR  86-01 

Yes  (1) 

No  (0) 

77  =  Data  Item  Descriptions 

Yes  (1) 

No  (0) 

78  =  CFE/CFAE  Notices 

Yes  (1) 

No  (0) 

79  =  AFADs 

Yes  (1) 

No  (0) 

80  =  SOW 

Yes  (1) 

No  (0) 


1  2.8% 

34  97.1% 


14  40.0% 

21  60.0% 


15  42.8% 

20  57.1% 


11  31.4% 

24  68.6% 


14  40.0% 

21  60.0% 


81  =  Did  the  commercial  manuals  require  additional  funding 

above  the  original  contract  projection? 

Yes  (1)  8  22.3% 

No  (0)  27  77.1% 

Section  Five 

82  =  How  did  the  program  office  or  maintenance  facility 

handle  the  standardization  of  parts  in  commercial 
systems? 

Submitted  TO  change  (2)  2  5.7% 

Went  back  to  the  contractor  to  correct  (1)  10  28.6% 

Other  (0)  23  65.7% 

83  =  Did  the  contractor  charge  for  corrections  required  in 

the  commercial  manuals? 

Yes  (!)  2  5.7% 

No  (0)  33  94.3% 


What  caused  the  lack  of  standardization? 

The  maintenance  facility  (2)  2  5.7% 

The  contractor  facility  (1)  11  31.4% 

Both  (3)  1  2.3% 

Other  (0)  21  60.0% 

Were  there  any  cases  of  the  actual  equipment 
configuration  not  matching  the  technical  documentation 
during  subsequent  repair  activities? 

Yes  (1)  10  28.6% 

No  (0)  25  71.4% 

Number  of  mismatches  found  in  the  commercial  manuals 
0-2  (1)  6  17.1% 

3-5  (4)  3  8.6% 

6-8  (7)  0  0.0% 

9-11  (10)  0  0.0% 

Over  11  (13)  5  14.3% 

No  response  21  60.0% 


Once  a  problem  was  solved,  was  a  supplement  or  TO 
change  accomplished? 

Yes  (1)  9  25.7% 

No  (0)  26  74.3% 


Section  Six 


Are  the  commercial  manuals  and  service  bulletins 
adequate  for  use  by  military  5  and  7  level 
technicians  in  repairing  the  equipment  to  the  same 
levels  the  AF  maintains  MIL-STD  equipment? 

Yes  (1)  14  40.0% 

No  (0)  21  60.0% 

To  what  extent  does  the  commercial  documentation 
require  supplementation? 

Under  5%  (.025)  1  2.8% 

6-10%  (.08)  0  0.0% 

11-20%  (.155)  1  2.8% 

21-30%  (.255)  3  8.6% 

Over  30%  (.31)  6  17.1% 

No  response  24  68.6% 

Are  sufficient,  in-depth  documentation  that  comply ' s 
with  the  system's  maintenance  concept  being  received? 
Yes  (1)  17  48.6% 

No  (0)  18  51.4% 


Was  the  contractor  required  to  provide  updates  and 
changes  with  commercial  equipment  modifications? 

Yes  (1)  19  54.3% 


92  =  Are  all  the  users  getting  the  service  bulletins  and 
interim  safety  supplements? 

Yes  (!)  14  40.0% 

No  (0)  21  60.0% 
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Appendix  B :  Raw  Data 

AOIMOIAOICJIWWAOIOIAAOlUOItilAMWWWUCJIWtilAMAOlUiaiaiH 

UOI(JIUUI^OMMA^AMfiK>N)OIO^MMUMK>tOOOJMWW04M(OrOK> 

0OM>i*OH)-»MO9C3Q<S(BCDH»^0»*S>Q<8A^fiQ»i*KKKQ0HHCD 
9  9  O  9  O  (D  (D  CD  9  A  O  A  A  A  A  A  A  A  A  A  A  A  A  A  A  A  ^  A  A  A  A 

^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^ 

AAAAAA^AAAAAAAAAA  A  AAAAA^AAn*i^AAAAAAA 

AAAAAhhAAAAAAAAAmAAAAA^hhAAkhkAAhAA 
A  A  A  A  ^D  A  A  A  A  A  A  A  A  A  A  A  A  A  ^  A  A  ^  A  A  A  A  A  A  A  A 

AAAMAAi-*«i*AAAAAAA^KAi-*AAA»«^AAAMAN*AA>*AA 
AAAHAA*i*AAAAAAAA^N»AAAAAAk*>*AAAN*AAAt*AA 
AAM»rfAAAAAAAAAAA^^A»Afl>g>(B^^AAAHMk*AA^AA 
9(0(rt^^^^{P{pQQQ(|^9QH^Ah*AAAHAHAHHHKAAHAA 
a  a  9  9  9  9  9  9  9  9  9  9  9  ^  ^  9  9  9  9  9  9  9  9  9  ^  9  9  9  9  9  ^  9  9 

9  9  9  9  9  9  9  9  9  9  99999m^99999m999999999m99 
999M99<^99999999*^H*99999M99999^999t^99 
99mh999m9999999mm99999m999^9m999m99 
9  9  9  9  9  9  9  9  9  9  99m9999999999999999999999 
99999m9h9mm999mmi^m99n*999m9k^^9h9m999 
9m9999mh*^mmm9m99m999m9mm9k9kh999m99 
9999999h9m999h9999m9m9k99k999m99mmi^ 
m9m9999h99999mmmmm9999^m999m9m99k99 
i'^99999^9^^f^999999999999999t^999i^9^99 
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VARIABLES 


Appendix  C :  Expanded  Responses  from  Questionnaire 

Actual  Survey  Response  To  Question  7 

Contractor  support  will  be  required  for  the  life  of  the 
system  for  non  stocklisted  items.  Numerous  Boeing  707 
parts  were  removed  from  salvaged  aircraft,  refurbished 
and  put  in  the  supply  system  because  of  limited  pai ts 
supply . 

Actual  Survey  Response  to  Question  14 

This  entire  area  of  reviewing  manuals  could  use  some  work 
in  my  opinion.  I  was  the  first  line  shop  supervisor  when 
the  commercial  manuals  were  sent  to  the  maintenance  shop 
for  a  maintenance  review.  We  had  two  days  to  review  over 
20  manuals,  some  4,000-^  pages  of  technical  material,  and 
needless  to  say  our  review  was  not  extensive  enough  to 
make  anything  but  a  very  general  statement  on  the 
adequacy  of  the  books.  A  better  method  would  require  the 
contractor  to  provide  a  deliverable  item  which  would  show 
a  comparison  of  the  CM  to  a  AF  manual  or  the  MIL-STD. 

Actual  Survey  Response  to  Question  19 

The  C-18A  aircraft  technical  orders  developed  by  Boeing 
received  an  in-process  review  and  pre-publication  reviews 
prior  to  delivery. 

No  quality  checks  are  being  made  to  the  standard 
commercial  manuals  {other  than  the  original  MIL-M-7298 
review)  although  the  manuals  receive  spot  checks  during 


i 


i 


4 


K 
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review  of  the  unique  information  which  is  being  added  to 
them.  The  unique  information  is  being  in-process 
reviewed,  validated,  and  verified.  These  commercial 
manuals  have  been  in  existence  since  1973  and  are  very 
accurate . 

Quality  of  commercial  manuals  is  checked  at  the 
commercial  manual  reviews.  In-turn,  this  determines  the 
adequacy  of  the  commercial  manual  for  field  usage.  (see 
MIL-M-7298C)  Commercial  manuals  are  not  validated  or 
verified,  however,  MIL-SPEC  supplements  to  commercial 
manuals  will  be  validated  and  verified. 

Actual  Survey  Response  to  Question  25 

The  commercial  manuals  are  divided  by  ATA-100  chapter 
into  separate  TOs. 

Quality  of  manuals  for  F-15  tactical  electronic  warfare 
support  system  (TISS)  were  checked  through  three  in- 
process  reviews.  A  quality  check  prior  to  preliminary 
publication  (ensuring  all  IPR  comments  were 
incorporated) ,  validation,  and  a  two  step  verification 
process . 

Actual  Survey  Response  To  Question  27 

The  IPBs  were  inadequate  in  that  the  illustrations  were 


not  detailed  enough  to  locate  bit  piece  parts  locations. 
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Actual  Survey  Response  To  Question  31 

Air  Force  Satellite  Control  Facility  (AFSCF)  Standard 
310-2. 

Actual  Survey  Responses  To  Question  32 

Manuals  were  validated  and  verified  in  the  usual  (and 
simplistic)  sense  of  TO  00-5-1.  However,  program 
integrated  support  required  the  manuals  to  be  included 
and  correlated  during  design  reviews,  audits, 
maintainability  demos,  test/evaluation,  etc. 

By  in-process  review  also. 

No  quality  check  was  performed.  However,  the  manuals 
were  reviewed  for  missing  portions,  i.e.  IPBs. 

Actual  Survey  Response  To  Question  46 

Cost  should  have  been  incurred,  but  SPO  refused  to  allow 
recommended  changes  to  be  made. 

Actual  Survey  Response  To  Question  47 

The  procurement  cost  of  the  commercial  manuals  was 
included  in  the  equipment  price.  However,  the  cost  to 
upgrade  them  to  MIL-C-7298C  was  not.  They  were  not 
upgraded,  so  no  costs  were  incurred. 

Yes--the  contract  only  covered  the  cost  of  the  commercial 
manual  submittals  and  CFE/CFAE  notices.  The  cost  to 
supplement/rewrite  and  purchase  additional  manuals  were 
covered  in  a  task  change  proposal  to  the  basic  contract. 
Added  contract  change  requirements  at  the  user's  request. 
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$587k  has  been  spent  to  modify  seven  commercial  manuals. 
This  program  is  still  in  lOT&S  and  the  manuals  have  not 
been  used  by  the  field  yet.  Changes  that  have  been  made 
are  the  result  of  verification  of  these  commercial 
manuals . 

The  commercial  manuals  for  the  IFF  subsystem  were  so 
inadequate  [that]  MIL-SPEC  TOs  were  procured  requiring 
additional  funding. 

Actual  Survey  Response  To  Question  48 

The  problem  was  not  solved  although  some  errors  were 
found  during  the  production  phase  were  corrected  by  the 
commercial  item  vendor  and  the  prime  contractor  at  no 
additional  costs.  The  reduction  of  operational 
dependency  on  commercial  publications  was  achieved  as  a 
program  objective,  so  we’re  not  using  the  manuals  enough 
to  detect  any  additional  errors.  However,  it  is  certain 
that  they  exist. 

Source  control  techniques  were  used  to  positively 
identify  the  series,  make,  model,  and  part  number  of 
commercial  items.  This  was  done  to  place  more 
responsibility  for  functional  configuration  on  commercial 
vendors  during  reprocurement  actions.  In  essence,  we  are 
saying  that  we  aren't  too  concerned  about  what  we  are 
sold  under  the  source  control  number,  as  long  as  it  is 
exactly  like  —  or  equivalent  to  —  the  original  item 
which  was  used.  Realistically,  this  equivalence  can  only 
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be  determined  by  the  vendor  and  then  only  if  we  can  tell 
the  vendor  precisely  what  was  originally  used.  This 
initiative  is  one  of  many  implemented  within  the  context 
of  our  totally  integrated  support  concept,  unusual  in 
systems  acquisition.  Our  program  objective  is  to  provide 
a  full-time,  operationally  suitable  tester.  To  do  this, 
we  accommodated  the  characteristics  of  commercial 
equipment  --  benefits  and  shortcomings  --  in  a  "customer 
oriented"  way  which  includes  all  aspects  of  our  military 
needs.  During  our  accommodating  process,  we  learned  that 
commercial  manuals  weren’t  required  in  any  but  two  roles; 
First,  as  source  data  for  relevant  military  manuals  and 
second,  as  a  form  of  data  useful  to  logistics  managers 
and  engineers  in  their  ongoing  materiel  management 
efforts . 

[These  was]  no  attempt  to  standardize  parts  on  commercial 
equipment . 

Actual  Survey  Responses  To  Question  50 

Commercial  carriers  modified  selected  components  to  fit 
their  mode  of  operation.  Normally  form  and  fit  were  the 
same  but  function  changed.  These  unique  items  have  to  go 
back  to  the  original  repair  facility  using  carrier  part 
numbers  to  be  repaired  to  obtain  replacement  parts. 
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Actual  Survey  Responses  To  Question  53 

There  were  two  incidents  where  the  users  discovered  that 
the  Illustrated  Parts  Breakdown  did  not  match  the  actual 
hardware.  However,  the  AFAD  required  the  contractor  to 
furnish  necessary  changes  throughout  the  life  of  the 
contract.  Therefore,  there  will  be  no  additional  cost  to 
the  government. 

Returned  to  original  contractor  support  facility. 

Actual  Survey  Response  To  Question  54 

By  submitting  a  COD  to  contractor  for  a  no  cost 
change/supplement.  (1  year  warranty  clause) 

Actual  Survey  Response  To  Question  56 

While  there  are  no  supplements  to  the  manuals,  the  AF 
unique  information  is  being  "rolled  into"  the  manuals  to 
reflect  the  unique  configuration.  This  action  was  a 
contractual  requirement  from  the  start. 

Actual  Survey  Responses  To  Question  60 

Commercial  equipment  publications  are  only  one  of  many 
factors  which  impact  commercial  equipment  support.  Our 
management  systems  are  designed  to  support  military 
equipment  and  are  adequate  if  we  adequately  accommodate 
commercial  equipment.  The  application  of  MIL-M-7298C  is 
one  part  of  this  "militarization."  Other 
"militarization"  actually  converts  commercial  equipment 
into  military  equipment.  If  done  well,  this  is  an 
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excellent  approach,  but  the  end  product  is  not  commercial 
equipment  —  a  point  lost  on  many  equipment  managers. 

Too  often,  our  approach  is  "militarization"  is 
little  beyond  a  rote  attempt  to  force  a  commercial  item 
into  the  military  operational  and  support  scene.  The 
results  are  partially  responsible  for  this  questionnaire. 
Militarization  of  equipment  which  is  wrong  for  its 
intended  use  doesn't  make  the  equipment  more  suitable. 
Similarly,  the  application  of  MIL-M-7298C  to  manuals 
which  are  wrong  for  their  intended  use  only  militarizes 
unsuitable  manuals. 

Responding  to  long-standing,  well  known  commercial 
equipment  support  problems,  this  program  accepted  an  ILS 
objective  to  achieve  suitable  support  for  its  commercial 
equipment.  As  implemented,  selection  of  commercial 
equipment  included  publications  reviews  and  support 
assurances  from  the  commercial  vendors.  In  other  words, 
as  the  customers,  we  explained  our  military  requirements 
to  the  vendors,  then  selected  those  most  sensitive  to  our 
needs.  This  approach  has  proven  more  or  less  effective, 
depending  on  the  size  of  the  vendor. 

We  have  encountered  some  blase'  attitudes, 
especially  among  the  large  vendors,  which  could  result  in 
support  problems.  However,  our  system  is  designed  to 
allow  form,  fit,  and  function  "substitutions."  So,  if 
support  problems  materialize,  then  other  vendors  will 


help  to  solve  them  through  normal,  commercial 
competition.  That  is,  if  AFLC  can  be  convinced  to 
implement  and  exploit  this  approach. 

On  this  program,  we  were  able  to  radically  reduce 
the  technical  manual  burden  by  using  commercial  manuals 
as  source  data  for  the  preparation  of  .modified  .military 
technical  orders.  "Modified"  means  that  the  manuals  were 
prepared  using  the  MIL-SPECs  as  guidelines,  but 
deviations  were  made  where  it  made  sense. 

We  were  able  to  eliminate  over  eight  linear  feet  of 
manuals,  all  of  various  sizes,  shapes,  costs,  and 
usefulness.  The  most  dramatic  reduction  involved  manuals 
for  the  computer  subsystem:  Over  six  linear  feet  of 
commercial  publications  costing  over  $5K  per  set  (28  sets 
were  procured)  were  reduced  to  only  a  few  pages  of 
operationally  relevant  matter  in  our  military  manual, 
deleted  data  wasn't  compatible  with  out  training,  skills, 
reading  comprehension,  AFTO  maintenance  methods,  system 
maintenance/support  capabilities,  and  etc.  so  it  was 
neither  relevant  nor  suitable. 

Also,  we  were  able  to  significantly  reduce  our 
support  equipment  burden.  Usually,  integrating 
contractors  simply  identify  (to  us)  whatever  SE  is 
identified  in  the  various  commercial  manuals  without 
regard  for  our  operational  situation.  We  required  the 
contractor  to  identify  only  that  SE  relevant  to  our 
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operational  and  support  requirements  --  as  defined  by  our 
integrated  support  efforts.  This  effectively  eliminated 
nearly  all  SE  for  the  commercial  equipment  we  use  since 
virtually  none  is  repaired  in  the  field  —  outside  of 
PMEL  and  PMEL  usually  has  an  existing  capability  and  it 
routinely  uses  commercial  publications  in  their  original 
form. 

Finally,  the  commercial  data  eliminated  at  the  field 
level  is  relevant  to  our  total  support  management  effort, 
but  as  a  form  of  "logistics  engineering  data"  or  data 
useful  for  materiel  management  and  support,  but  neither 
formal  engineering  data  nor  technical  manuals.  To  date, 
a  AFLC  dogma  has  refused  to  consider  the  data  under  such 
a  category,  choosing  instead  to  force  the  data  into  the 
AFTO  system  where  it  will  languish  and  evolve  its 
obsolescence. 

To  4950  TESTW/MASV  is  to  TOMA  for  the  EC-18B  aircraft, 
SMILS,  OPTICS  and  ECCM/Advanced  Radar  Test  Bed  Aircraft. 
We  also  supervise  the  TO  development  for  selected  ASD 
System  Program  Offices.  The  survey  was  completed  based 
on  the  American  Airlines  (AA)  Boeing  707  aircraft 
purchased,  in  FY82,  by  ASD/AFY  for  the  4950th  Test  Wing. 
This  purchase  was  unique  since  the  aircraft  were  used  and 
purchased  from  the  carrier  (AA)  with  existing  commercial 
data  supplied. 
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The  technical  data  for  the  707  aircraft  had  been 
developed  to  Air  Transport  Association  (ATA)  100, 
revision  3  specifications.  Two  hard  copies  and  numerous 
film  cartridges  were  delivered  with  the  aircraft  along 
with  film  cartridges  of  Engineering  Change  Orders  (ECO) , 
Engineering  Service  Orders  (ESO) ,  and  Flight  Directives 
(FD) .  The ■ commercial  aircraft  manuals  were  to  be  entered 
into  the  Air  force  technical  order  Systems  and  maintained 
by  an  engineering  contract.  Since  Boeing  did  not 
maintain  the  data  for  AA,  they  would  not  verify  that  the 
data  was  accurate;  therefore,  new  maintenance  manuals 
were  developed. 

Boeing  used  their  existing  707  maintenance  data  and 
incorporated  the  AA  ECOs  and  FDs,  which  have  been 
accomplished  on  the  aircraft.  They  also  incorporated  some 
unique  AF  data  i.e.,  fuel  systems  maintenance/aircraft 
grounding.  In  addition  to  the  new  technical  orders,  the 
AA  ESOs  are  still  being  used  to  perform  some  off- 
equipment  maintenance.  ESOs  are  changes  to  commercial 
maintenance  instructions  required  by  carrier  unique 
modifications.  ESOs  can  also  be  driven  by  service 
bulletins  selected  for  accomplishment  by  the  carrier. 

The  ESOs  do  not  have  technical  order  numbers  affixed, 
since  the  quality  of  the  data  is  unknown.  The  only  way 
to  determine  the  accuracy  of  the  ESOs  is  to  perform  a 
verification.  OC-ALC  is  reviewing  the  ESOs  being  used  by 
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the  Test  Wing,  trying  to  determine  how  to  approach  the 
SSO  problem.  Presently,  there  is  no  method  to  get 
changes  into  the  ESOs  other  than  at  user  level.  No 
provisioning  was  accomplished  for  the  aircraft,  after 
purchase,  so  the  maintenance  personnel  work  the 
discrepancies  as  far  as  possible  and  use  the  CLS  contract 
when  it  goes  beyond  our  maintenance  capability;  Since 
the  707  has  numerous  items  which  are  stock  listed  in  the 
supply  system,  existing  technical  orders  are  used  to 
perform  intermediate  level  maintenance  when  available. 

For  of  the  eight  C-18A  aircraft  were  modified  to  EC- 
133  aircraft  by  the  4950th  Test  Wing  Modification  Center. 
OC-ALC  required  that  "stand-alone”  aircraft  maintenance 
manuals  be  developed  to  operate  and  .maintain  the  aircraft 
after  mod.  The  C-18A  aircraft  manuals  were  used  as 
baseline  technical  information  and  the  ARIA  engineering 
data  package  was  used  as  source  data  by  Boeing  to  develop 
the  EC-18B  aircraft  technical  orders.  Flight  manuals  and 
maintenance  manuals/illustrated  parts  catalogs  have  been 
developed  for  the  EC-18B  aircraft  configuration.  An 
update  is  required  to  the  technical  orders  because  of  the 
lag  in  engineering  data  completion  by  the  Mod  Center. 
Based  on  latest  estimates,  update  will  occur  in  FY91/92 
time  frame. 

4950th  Test  Wing/MASV  is  also  involved  in  the 
commercial  manual  world  on  aircraft  programs  being 
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managed  by  ASD.  Here  again,  the  manuals  are  in  ATA  IOC 
specification  developed  for  the  "green”  aircraft 
purchased.  Normally,  GFE  or  GFE  compatible  equipment 
such  as  UHF  and  HF  radios,  IFF,  TACAN,  and  INS  are 
installed  prior  to  delivery.  The  commercial  world 
includes  the  item  equipment  manual  as  the  maintenance 
instructions  in  the  "green"  aircraft  manual,  along  with  a 
wiring  interface,  but  no  troubleshooting  instructions  for 
newly  installed  equipment.  For  Air  Force  use, 
maintenance  instructions  and  aircraft  interface  for  these 
systems  are  normally  developed  as  supplemental  data,  or 
may  be  folded  into  the  commercial  manuals  prior  to 
delivery.  Since  the  commercial  inspection  system  is  not 
used  in  the  military,  the  inspection  criteria  is  nor.mally 
generated  by  the  contractor  in  MIL-SPEC  requirements, 
flight  manuals  and  check  lists  are  developed  from  the 
commercial  flight  manual  but  are  formatted  to  MIL-M-TTOCt 
Specs . 

Commercial  aircraft  maintenance  manuals  are  adequate 
for  use  by  Air  Force  technicians.  The  main  problem  is 
learning  how  to  find  the  information  in  the  manuals, 
since  maintenance  efforts  are  not  normally  duplicated  for 
performing  check-outs.  The  organizational  maintenance 
manuals  and  illustrated  parts  catalog  (IPC)  also  contain 
maintenance  instructions  and  parts  breakdown  for  off- 
equipment  tasks  which  are  considered  by  the  Air  Force 


intermediate  and  depot  level  maintenance.  For  the  4350th 
Test  ’»in5,  this  is  good  because  we  are  not  restricted  to 
the  level  of  work  we  can  accomplish. 

The  procurement  of  commercial  manuals  uses  MIL-M- 
7298  as  evaluation  criteria.  Amendment  5  to  this 
specification  requires  source  maintenance  and 
recoverability  codes  to  be  established  for  the  IPC.  The 
spec  also  requires  that  the  commercial  instructions  are 
to  be  reviewed  prior  to  purchase  for  usability  by 
acquiring,  using  and  supporting  commands.  This  is  all 
well  and  good  for  items  as  specified  in  MIL-M-7298C; 
however,  it  is  impossible  to  comply  when  purchasing 
aircraft  operation  and  maintenance  manuals,  because  of 
the  size  of  the  effort  to  provision  the  aircraft  and  the 
thousands  of  pages  of  technicalnical  information  supplied 
to  review.  One  additional  item  of  interest  is  the  use  of 
ATA  100  specifications.  ATA  100  was  issues  1  Jun  56. 
Since  that  time,  the  spec  has  been  revised  25  times.  The 
commercial  world  views  the  spec  as  a  guideline  to  follow- 
-not  how  to  do  the  work,  which  is  what  we  use  the  MIL- 
SPEC  for.  Therefore,  it  is  hard  to  pin  a  contractor  down 
on  format  and  procedure  arrangement.  Also,  the 
contractor  may  have  updated  the  manuals  for  old  aircraft 
which  are  still  in  production — thus,  not  all  manuals  are 
made  to  the  same  spec  revision. 
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In  conclusion,  the  commercial  manual  world  for 
aircraft  varies  depending  on  who  is  getting  the  aircraft 
and  the  maintenance  concept  for  deployment.  Since  the 
commercial  off-the-shelf  purchase  of  aircraft  is 
relatively  new,  mistakes  made  in  the  technical  manual 
area  during  acquisition  will  not.  be  known  for  several 
years.  A  possible  solution  is  to  warranty  the  commercial 
manuals  for  a  specified  period  of  time  after  purchase, 
since  the  off-the-shelf  aircraft  acquisition  norm.ally 
spans  a  short  time  period  and  aircraft  PMUT  before  the 
technical  manuals  are  completed.  The  warranty  would 
require  the  contractor  to  make  specific  changes  if 
certain  procedures  were  discovered  unusable  by  government 
personnel,  or  did  not  match  hardware  configuration. 

As  far  as  I  can  see,  the  AF  will  have  few,  if  any, 
problems  with  the  commercial  data  being  procured  in  this 
program.  The  technicians  who  will  use  this  data  are  all 
very  skilled  and  experienced  people.  The  data  could  also 
be  usable  by  3-  and  5-  level  technicians  if  they  were 
given  any  real  training.  The  writers  of  these  manuals 
assume  that  the  person  using  them  is  an  experienced 
technician  and  writes  the  manuals  accordingly.  These 
manuals  appear  to  be  very  accurate  and  complete. 

The  reason  so  many  questions  were  left  blank  is:  we  are 
early  in  FSD  and  not  that  many  manuals  have  been  written 
to  date.  We  have  not  had  a  chance  to  review  any 
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ccn'.mercial  data,  we  have  only  review  changes  to  existing 
Group  A  technical  data. 

If  ccainercial  manuals  exist,  use  them.  Please  don't 
rewrite  to  a  relaxed  -1  format  just  to  have  an  official 
looking  TO!  In  the  Automatic  Test  Equipment  area  we  see 
a  wide  range  of  manuals,  the  normal  operation, 
maintenance,  overhaul  and  IP3  for  hardware  along  with 
operator  (user)  ,  programming  and  language  .manuals  for 
software.  Most  are  very  good.  Often  .much  technical 
information  is  lost  when  commercial  .manuals  are  .m.olded  in 
the  MIL-M-33734  format.  We  must  keep  in  mind  "why  we 
need  technical  manuals  and  the  purpose  they  are  to 
serve."  Marketing  tendency  to  be  optimistic  is  stating 
equipment  performance  and  specmanship  .must  be  filtered 
out  of  our  number  of  computer  .manuals  written  by  the 
English  Department  that  cause  much  confusion  and  waste, 
technical  accuracy  is  a  must.  Provide  technical 
information  directly  and  clearly. 

One  electrical  drawer  the  TRR  receiver,  was  being 
verified  when  it  was  realized  that  it  was  impossible  to 
remove  two  of  the  LRU's.  This  problem  required  changes 
to  the  TM  to  the  level  of  sparing.  Provisioning  had  been 
accomplished  and  sparing  ordered,  new  spare  items  where 
ordered  and  old  ones  where  canceled.  Life  cycle  cost 
figures  had  to  be  changed  and  R&M  figures  where  also 
effected . 
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This  reply  is  for  the  Comfy  Sword  III  Phase  2  {CS  III-2) 
program.  CS  III-2  is  a  one  of  a  kind  system  that  will  be 
used  at  Nellis  Range  and  will  have  CLS  maintenance  for 
the  life  of  the  system.  Our  program  has  both  commercial 
manuals  and  MIL-STD  manuals.  Commercial  manuals  were 
submitted  under  CFS  notices,  which  were  reviewed  and 
approved  through  the  normal  CFS  approval  process  by  SN- 
ALC.  We  intend  to  verify  the  commercial  manuals  lAW  TO 
00-5-1  at  the  same  time  as  we  verify  the  MIL-STD  manuals. 
The  contractor  is  in  the  process  of  validating  the  MIL- 
STD  manuals;  the  government  will  then  verify  them. 

Because  CS  III-2  has  not  yet  been  turned  over  to  the 
user,  it  is  not  clear  at  this  time  what  problems  there 
will  be  with  commercial  manuals.  We  have  had  at  least 
one  manufacturer  of  our  commercial  equipment  go  out  of 
business,  which  also  caused  a  problem  in  obtaining 
manuals.  We  also  should  have  but  did  not  have  the  user 
involved  in  review  and  approval  of  commercial  manuals. 
IPTS  is  a  quick  pace  program.  We  are  having  the 
contractor  convert  commercial  manuals  to  MIL-SPSC 
manuals.  the  contractor  is  in  the  process  of  writing  the 
manuals  so  I  could  not  answer  some  of  your  verification 
questions  since  we  haven't  started  yet.  One  of  the 
problems  I  have  with  contractors  is  over  optimism.  They 
always  believe  their  manuals  have  no  problems  and  will 
easily  pass  verification  and  thus  allow  no  time  for 
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correction  and  follow-on  verification.  No  prepublication 
is  ever  factored  into  their  tir.e  lines  or  time  for  SN-.\LC 
to  accomplish  printing.  Manuals  are  inevitably  delivered 
30  days  prior  to  user  need  not  taking  into  account  these 
above  factors.  Although  the  contractor  feels  he  has  met 
the  requirement  manuals  are  still  not  available  to  the 
field.  If  it  wasn’t  for  slips  in  schedule  unrelated  to 
logistics  manuals  would  be  late  and  thus  delay 
support ability . 

I  can’t  provide  much  information  on  Sentinel  Bright 
commercial  manuals.  Most  of  the  work  was  completed  prior 
to  my  arrival  in  IC .  But,  I  can  tell  you  that  the  intent 
is  to  support  the  system  with  contract  support  for  the 
life  of  the  system.  We  have  updates  and  service 
bulletins  subscription  to  supplement  the  co.mmercial 
.manuals.  We  feel  this  will  aid  in  making  the  contract 
support  contract  more  competitive.  Most  of  the  equipment 
is  commercial  off-the-shelf. 

Most  commercial  manuals  do  not  provide  the  kind  of 
information  required  to  perform  maintenance  in  accordance 
with  the  assigned  SMR  code.  In  other  words,  piece  part 
repair  of  SRUs  cannot  adequately  be  performed  using 
commercial  manuals  (in  many  cases).  Proprietary  data 
rights  is  often  the  obstacle  to  acquiring  more  detailed 
information.  Cost  is  also  a  big  problem.  With  so  much 
COTS  equipment,  we  m.ust  be  smarter  in  our  handling  of 
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technical  data.  We  in  OTH-B  have  attacked  the  prohle.-n 
froj.  several  angles  to  keep  costs  down. 

(1)  MIL-SPEC ' ing  as  few  manuals  as  possible. 

(2)  Use  commercial  manuals  as  is--buy  missing 
information  to  bring  manual  up  to  acceptable  maintenance 
level  (without  worrying  about  729SC  format) . 

(3)  Revisiting  SMR  code  assignment — Is  it 
realistic? 

(4)  Buying  additional  manuals  directly  from  the 
vendor.  Don't  fight  copyright  laws  just  so  government 
can  reproduce  manuals. 

(5)  See  if  ALCs  can  supplement  manuals.  The  ALCs 
have  a  wealth  of  talent  in  their  MA  shops. 

(6'  Combining  similar  commercial  manuals  into  one. 
Commercial  manuals  are  generic  and  do  not  always  show 
differences  in  models  of  their  product.  The  equipment  is 
difficult  to  maintain  when  a  technician  must  use  a  MIL- 
SPEC  supple.ment  cross  referenced  to  a  commercial  manual. 
For  example,  the  MILTOPE-MSU  and  the  ITT  supplement.  A 
mix  of  commercial  manuals  and  MIL-SPEC  for  a  single  item 
should  be  avoided. 

Some  valuable  lessons  learned  have  been  experitnced  on 
this  program  as  a  result  of  using  65%  COTS  with  some  62 
CFAE/CFE  notices  generated. 
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Need  to  assure  the  integrating  contractor  reviews 
all  cor.r.ercial  manuals  for  preventive  maintenance 
inspection  (PMI)  requirements. 

Need  to  review  the  manual  to  assure  the 
troubleshooting  procedures  go  down  to  the  piece  part 
level  for  depot  maintenance. 

In  the  case  where  several  COTS  items  have  been 
integrated  into  a  ccmplete  unit  has  the  integrating 
contractor  been  tasked  to  develop  troubleshooting 
procedures  to  get  you  to  the  individual  commercial  item. 

How  do  you  apply  new  terms  generated  by  the  FAS.  when 
your  contract  was  let  under  the  DAR  and  these  terms  had  a 
different  meaning?  i.e.  Modified  COTS  vs.  custom 
product  or  modified  COTS  vs.  commercial  type? 

This  program  has  not  been  able  to  get  a  very  good 
response  from  the  ALC  on  commercial  manuals  through  the 
CFAE/CFE  notice  procedures.  Many  of  our  commercial 
manuals  have  been  at  the  ALC  for  18-24  months  with  no 
response  and/or  approval/disapproval.  Commercial  manuals 
have  always  been  a  problem  and  will  continue  to  be  a 
problem  until  we  move  the  management  of  technical  data 
under  '^ne  roof  (AFLC  vs.  AFSC.) 

The  entire  subject  of  commercial  manuals  for  repair  of 
Air  Force  assets  is  to  large  to  addressed  here  with 
specifics.  Manuals  provided  by  a  contractor  are  as 
varied  in  scope  and  content  as  the  contractors 
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themselves.  Since  we,  the  Air  Force,  deal  mainly  with 
the  largest  corporations  on  acquisitions  for  weapon 
systems  a  standard  for  acceptable  manuals  would  not  seem 
so  difficult  a  problem  until  you  begin  to  look  at  the 
multitude  of  small  subcontractors  that  provide  parts  of 
the  system  to  the  prime  contractor.  The  problem  is 
further  exacerbated  by  contract  law  that  I  really  do  not 
have  the  expertise  to  address  but  I  do  know  there  are 
"cans  and  cannots"  associated  with  every 
specification/requirement  you  levy  on  a  contractor.  My 
personal  opinion  is  that  there  is  no  single  answer,  or 
even  multiple  answers.  The  overall  solution  will  be  a 
process  whereby  we  will  adapt  to  commercial  manuals  as 
best  we  can,  develop  the  necessary  data  where  we  must  and 
buy  MIL-SPEC  TOs  when  there  is  a  necessity  to  do  so. 

The  F103-CF-100  engine  Intermediate  Maintenance 
Preliminary  technical  Orders  in  MIL-SPEC  format  were 
delivered  to  the  government  in  Sep  84.  Verification  of 
these  manuals  has  begun.  The  Illustrated  Parts  Breakdown 
in  MIL-SPEC  format  is  due  to  be  delivered  for 
verification  15  Oct  84.  The  depot  Maintenance  Manual  set 
in  MIL-SPEC  format  is  scheduled  for  50%  In  Process  Review 
23  Oct  through  1  Nov  84. 

Past  experience  with  JT3  engine  commercial 
maintenance  manuals  show  the  cost  to  be  approximately 
$700  per  manual.  CFMI  has  pointed  out  in  the  past  that 
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only  the  first  2C  each  preliminary  copies  of  the 


commercial  manuals  would  be  furnished  "free"  of  cost  to 
the  government.  Future  copies  and  future  change  costs 
would  have  to  be  negotiated.  Initial  distribution  for 
existing  engine  depot  maintenance  manuals  is 
approximately  300  copies.  Cost  of  updating  a  MIL-SPEC  TO 
is  an  average  of  $200  par  page  on  a  TO  update  contract. 
Using  that  cost  percentage,  we  don’t  understand  the  high 
cost  quote  to  ASD  unless  it  is  for  development,  not  just 
a  changed  page. 

Thera  are  other  expenses  which  would  be  incurred  as 
result  of  a  decision  to  use  commercial  manuals  for  the 
depot  repair  manual.  Our  maintenance  personnel  are  not 
all  well  trained  or  experienced  as  co.Tjmercially  certified 
mechanics.  This  would  require  training  and/or  in-house 
supplementation  as  a  result  of  less  detailed  manuals. 
Costs  of  training  and  supplements  have  not  been 
considered  in  ASD ' s  cost  figures.  Supplements  would  also 
be  necessary  to  furnish  Air  Force  only  data  which  would 
not  be  included  in  the  commercial  set  of  manuals. 

Examples  of  these  would  be  nuclear  hardening,  corrosion 
control  procedures,  and  comprehensive  engine  management 
system  data.  The  commercial  manuals  do  no  generate 
different  limits  for  military  versus  commercial 
customers;  therefore,  engine  repair  concepts  do  not 
differ.  If  any  changes  become  necessary,  they  would  be 
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incorporated  as  supplements.  Supplementation  increases 
the  cost  of  the  commercial  manuals  and  leads  to  errors  in 
the  manuals  as  changes  are  made  which  affect  other  parts 
and  makes  use  of  the  technical  data  by  shop  personnel 
very  difficult. 

Additionally,  the  commercial  Illustrated  Parts 
Catalog  {IPO  contains  all  airline  ad  Air  Force 
configurations.  A  training  program  would  be  necessary  to 
instruct  Air  Force  personnel  how  to  use  the  IPC.  The  IPC 
also  does  not  include  source  maintenance  recoverability 
(SMR)  codes  for  the  field  to  know  the  level  of  repair  or 
spare  parts  authorization.  On  the  other  hand,  the  MIL- 
SPEC  Illustrated  Parts  Breakdown  (IPB)  would  facilitate 
Air  Force  control  of  standard  DOD  engine  parts; 
otherwise,  commercial  sole  source  procurement  would  be 
necessary.  Commercial  airline  changes  will  continuously 
be  added  to  the  commercial  manuals  and  would  always  be  a 
point  of  confusion  to  maintenance  personnel. 

The  commercial  overhaul  manuals  are  broken  out  by 
component  and  contain  multiple  processes,  assembly, 
disassembly,  repair  and  inspection  procedures  in  the  same 
section.  OC-ALC  maintenance  concept  is  setup  by  work 
station.  Each  station  has  a  specific  task  to  perform. 

The  MIL-SPEC  work  package  TO  was  developed  for  this 
concept.  Only  work  packages  necessary  for  each  task 
performed  at  a  work  station  need  be  filed  at  the  work 
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station.  Sach  work  package  consists  of  a  title  page  and 
data  necessary  to  complete  the  task.  Commercial  manuals 
do  not  have  a  title  page  for  the  sections,  thus  cannot  be 
broken  out  by  tasks  and  placed  at  work  stations.  Using 
the  work  package  concept,  only  an  average  of  25  pages 
need  be  kept  at  a  work  station.  if  a  commercial  manual 
is  used  the  whole  TO  would  be  kept  within  a  shop  area 
(approximately  50,000  pages  at  each  work  station  would  be 
impractical)  and  the  worker  would  have  to  go  away  from 
his  work  station  to  the  manual  to  look  up  information. 
This  is  costly  in  the  time  a  task  is  to  be  completed  and 
will  impact  the  amount  of  time  spent  within  the  shop. 

Further,  CMFI  has  stated  in  the  past  if  a  USAF 
repair  was  submitted  for  inclusion  and  they  (CMFI)  did 
not  agree  with  it,  they  would  incorporate  the  change. 
However,  they  would  disclaim  it  and  note  it  as  USAF 
repair  only.  This  would,  consequently,  absolve  any 
warranties  the  USAF  might  have. 

OC-ALC/MMS  feels  that  dollar  savings  now,  if  any, 
would  be  lost  during  a  future  technical  data  conversion 
and  could  be  further  impacted  by  the  requirement  to 
manage  the  commercial  data  on  a  peculiar  basis. 

It  has  been  the  desire  of  OC-ALC  in  the  past  to 
maintain  a  high  degree  of  commonalty  between  the  FlOl-GS- 
102,  FllO-GE-100,  and  the  FlOS-CF-100  engines  in  order  to 
reduce  logistics  support  costs  and  improve  support 
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management.  The  same  applies  to  technical  data.  FlOl 
and  FllO  manuals  will  be  in  MIL-SP2C  format,  .pa 

Wartime  surge  will  be  adversely  affected.  The  F108 
will  probably  be  co-located  with  various  USAF  engines 
covered  by  MIL-SPEC  TOs.  Because  of  the  extreme 
differences  in  the  commercial  vs.  MIL-SPEC  TOs  it  would 
be  hard  to  put  personnel  onto  this  engine  and  expect  good 
results  for  the  surge.  This  point  should  not  be  taken 
lightly. 

CFMI  representatives  have  been  questioned  as  to  the 
Reading  Grade  Level  (RGL)  on  commercial  manuals,  they 
report  it  to  be  about  14.  MIL-SPEC  format  TOs  are 
structured  to  9.1  RGL  and  this  difference  is  significant 
when  we  consider  the  maintenance  personnel  are  for  the 
most  part  high  school  graduates. 

SPO  decision  to  CLS  on  contract  precluded  contracting  for 
logistics  deliverables  to  support  using  command 
maintenance  concept  or  AFLC  depot  level  support. 
Additionally,  the  lack  of  maintenance  data  precludes 
competitive  repair  contracts  for  depot  level  support 
which  results  in  sole  source  maintenance  contracting  to 
the  prime  contracts.  Lack  of  logistics  data  precludes 
future  changes  to  the  maintenance  concept  lAW  using  and 
supporting  commands  desires  based  on  cost  life  cycle 
estimates.  this  "worst  case"  support  scenario  gets 
worse. . .prime  contractor  refuses  to  bid  on  a  maintenance 
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contract  with  S?0.  Additionally,  runiors  indicate  prine 
contractor  is  to  declare  chapter  13  bankruptcy  which  wil 
result  in  a  non-supportable  program.  No  data  precludes 
release  of  contract  to  other  agencies.  In  the  event  of 
bankruptcy — contractor  is  required  to  provide  available 
data  to  government.  However,  pri-me  contractor  is  an 
integrating  contractor  who  had  not  developed  data,  but 
relied  on  vendors  for  equipment  deliveries.  Therefore, 
data  may  never  be  available - 

All  the  above  scenario  is  based  on  independent  SPO 
decision  for  a  support  concept  (CLS)  without  a  higher 
reviewing/approving  authority.  No  single  agency  (Air 
Staff)  is  tasked  to  approve  CLS  decisions  bases  on  LCC 
models . 

Historically,  HQ  SD  used  contractors  for  their  logistics 
support  of  all  their  systems.  The  general  feeling  from 
them  is,  AFLC  requirements  are  to  expensive  and  they  do 
not  understand  our  kind  of  operations.  Thus  they  see 
logistics  command  as  draining  their  program  dollars 
without  any  thought  that  LCC  would  be  reduced  if  our 
needs  are  met.  They  continually  refuse  our  inputs  and 
our  recommendations.  My  personal  concern  is  so  great  I 
feel  we  must  have  an  audit  to  surface  the  problems  we 
have  encountered  so  far. 


Conmercial  equipment  poses  so  unique  problems,  but 
not  insurmountable.  Organic  depot  can  repair  COTS,  but 
we  must  be  aware  of  the  pitfalls: 

(1)  Full  documentation  may  not  be  available 

a.  Level  2  engineering  data  can  be  used  to 
complete,  but  only  for  form-fit-function. 

b.  Configuration  control  still  cannot  be 
performed  but  we  may  request  level  3  data  on 

the  follow-on  contract,  if  we  find  it  is  cost 
effective  or  the  system  is  of  a  highly 
critical  nature. 

2.  TOs  {a  subset  of  engineering  data)  may  back  in 
completeness,  but  repair  can  still  be  affected. 

3.  Spares  may  not  be  available  for  sometime  but  we 
can  perform  some  type  of  spares  allocation.  Even 

COTs  contain  many  parts  that  are  stocklisted,  so 
the  provisioning  effort  may  not  be  that  large. 

The  bottom  line  is,  SPO  must  have  logistics  as  one 
of  his  major  concerns  and  have  responsibility  for  LCC  of 
system  they  are  bringing  into  the  inventory. 

COTS  design  is  based  on  market  factors  (profit, 
marketability) . 

Applicability  to  government  application  is  based  on 
commercial  specification,  therefore,  little  to  no 
government  influence  on  system  specification/design  for 
specific  military  applications. 
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COTS  equipment  software  also  developed  for  commercial 
applications  to  meet  market  factors  (profit,  sales...) 

—  Also  little  government  -nfluence  on  COTS  software 
Unique  military  requirements  drive  modification  of  COTS 
software  or  development  of  military  unique  software. 

Military  can  influence  design  of  only  military 
portion  of  software  modification — not  influence 
commercial  COTS  software. 

Military  can  obtain  support/data  for  militari- 
modification  portion  of  the  software  modification  and 
control  software  configuration  of  that  military 
modification  portion  only  the  limited  data  rights  for 
future  configuration  control,  is  limited  by  fact  that 
contractor  changes  to  COTS  software  may  make  military 
modification  portion  unusable — military  must  then 
redevelop  COTS  portion  of  software  due  to  co.mmercial 
discontinuation  of  soffware  version,  or  military  is 
forced  to  buy  software  version  life  cycle  support. 

—  Military  developed  software/equipment — full  data 
rights  in  future  life  cycle  support  if  support 
data/compatibilities  bought,  ease  of  configuration  control 
for  future  system  mode,  changes,  replacement  programs. 

COTS  equipment  changes  design  by  market  forces — no 
military  stability. 
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Military  modified  or  military  uniquely  developed 
software  is  different  for  CCTS  software  developed  for  th 
COTS  equipment. 

Military  modifications  and  military  unique  software 
developed  for  meeting  a  specific  military  application  is 
developed/designed  to  operate  on  specific  (generally 
COTS)  equipment. 

Changes  in  COTS  (contractor  terminated  product  line, 
changes  product  line,  or  replaces  it  with  a  .".ew 
version/si.milar  product)  may  or  may  not  have  backward 
compatibility  to  operate  the  prior  version  of  COTS 
software.  Also,  military  unique  software  or  COTS 
software  tailored  for  a  unique  military  application,  .may 
also  not  operate  on  new  COTS  equipment  replacing  the  old 
versron . 

If  COTS  equipment  the  military  software  operates  on 
is  terminated  by  the  vendor  (terminates  product  line), 
guarantee/warranty  clauses  in  contract  my  cover  only 
military  paid  for  modifications  to  the  software.  Cost  o 
COTS  software  data  rights  (terminated  by  contractor)  may 
be  prohibitive  (no  life  cycle  sparing,  data  for 
reprocurement,  etc.  =  insupportable  even  though  data 
offered  to  government.) 

New  replacement  COTS  may  not  be  able  to  operate 
military  software  developed  for  prior  version  of  COTS 
equipment . 
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Saves  ^.aintaining  older  version  of  COTS  equip:aent 
through  life  cycle  sparing,  buying  data  rights,  etc.= 
high  cost  in  order  to  maintain  system  and  configuration 
control  to  that  baselined  system. 

If  numerous  changes  to  system  using  the  COTS  are 
anticipated  it  doesn’t  make  sense  to  maintain  an  old 
system  configuration. 

New  version  equipment  would  not  have  been  designed  to 
use  military  unique  software  or  prior  COTS  software 
modified  for  military  application.  New  COTS  developed  to 
use  COTS  software. 

Forces  buying  new  COTS  equipment  with  COTS  software, 
and  either  redevelopment  of  the  COTS  software  for 
military  unique  application  {.modify  COTS,  or  develop  new 
military  COTS) --all  over  again. 

Military  in  some  position  over  a  barrel-“no 
configuration  control,  [andl  money  to  contractors. 

-  COTS  or  not  COTS  (AFR  300-21,  chapter  7) 

Commercially  available  equipment  (COTS) 

Meets  military  requirement 

Includes  both  software  and  equipment  as  available  for 
military  requirement 

No  modification  of  equipment  on  software  and  firmware 

Military  has  no  influence  on  design  or  perf ormance-- 
take  "as  is" 

Limited  to  no  data  rights  to  military 
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No  control  of  COTS  equipxenc/sof tware  changes 
Military  r.ust  renain  currant  in  COTS  configuration 
for  equipir.ent/sof tware 

Increased  likelihood  of  CLS  for  support  to  pricie 
manufactures  due  to  limited  data  availability  for 
competitive  requisitions.  All  support  dependent  o.n  pri.me 
contractor,  can't  compete  spare  due  to  lack  of  iaoa  = 
violation  of  FAR/public  law  possible. 

MOTS  (Military  off  the  shelf  equipment) 

COTS  equipment  and/or  software  modified  for  military 
requirements . 

Data  on  modification  paid  by  military  only 
Configuration  control  design  or  performance  of  COTS 
but  military  modification  portion  alone  —  limited 
design/performance  influence  by  military. 

Military  still  subject  to  contract  for  support 
Military  developed  equipment 
Full  data  available 

Military  can  control  design/performance 
Lift  support  available  to  meet  changing  military 
requirements 

NOTE:  Fer  AFR  300-21,  Chapte^'  7  under  review,  any  change 

to  system  design  or  performance  (including  software)  make 


the  system  non-COTS . 


Appendix  D:  Expanded  Response  from 
General  Electric  Interview 

General  Electric  Response. 

Q  Is  the  commercial  side  of  the  house  moving  a  head  more 
rapidly  in  automation  then  the  military? 

A  Absolutely.  The  commercial  side  of  the  house  has  a  data 
base  to  customize  data  (core  data  base.)  We  code  all 
data  by  engine  model,  aircraft  it  is  used  in,  and 
configuration  level  by  service  bulletin  number.  It  can 
retrieve  a  customized  version  at  anytime.  For  example, 
the  new  CFSS2  engine  (basic  family)  has  10  models,  is 
used  in  4  different  aircraft,  and  has  15  different 
customers.  Through  task  nu.T»bering,  a  system's  engine  and 
aircraft  is  built  into  the  data  base.  This  enables  the 
airline  to  have  a  shelf  support  system  or  a  line 
maintenance  support  system  based  on  the  task  nu.“ber3 . 
(This  data  base  does  not  tie  into  the  LSA  data  base.) 

The  task  number  identifies  the  part  of  the  engine, 
module  of  the  engine,  piece  part  of  the  engine  through 
the  first  three  numbers.  Then,  it  goes  on  and  tells  the 
function  (repair  or  disassembly)  in  repair  and  the  type 
of  repair  (weld,  etc.) 

Then,  each  industry  sets  up  another  data  base  and 
feeds  in  their  reliability  and  maintainability  data  that 
will  compare  to  some  LSA  element.  This  R  &  M  data 
identifies  the  skill  level,  consumerable ,  task  level, 
etc.  There  is  a  requirement  in  the  commercial  world  to 
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niaintain  data  basas  and  provide  isaijnetic  tape  to  the 
airlines . 

Currently  the  commercial  world  is  working  on  a 
compact  disk  and  real-time  network. 

When  dealing  with  the  military,  we  [GE]  must  create  a 
new  data  base  because  the  military  wants  special  formats 
(2  columns  vs.  full  page.)  If  the  military  were  to  use 
this  system,  we  could  add  the  Air  Force  unique  data 
through  the  existing  data  base. 

Q  The  problem  with  the  F-108  engines  was  mostly  due  to  the 
format  of  the  manuals? 

A  Mostly,  the  overhaul  shop  (depot)  OC-ALC  absolutely 
refused  to  except  the  commercial  bocks.  Therefore,  we 
ended  up  with,  even  though  it  was  sold  as  a  commercial 
progra.m,  the  parts  and  provisioning  being  done  in  a 
commercial  way;  while,  the  publications  were  done  in  a 
military  for.mat.  There  were  some  differences  in  the 
engines,  but,  it  was  felt  that  the  depot  drove  this 
decision . 

Q  To  supplement  the  F-103  engine  publications,  if  they 
would  have  stayed  with  he  commercial  books,  how  would 
you  have  gone  about  doing  that? 

A  I  don't  precisely  remember  the  recommendation.  But, 

here's  a  navy  example:  We  have  a  program  where  there  are 
available  commercial  manuals  covering  accessories. 
Wherever  there  is  no  difference,  the  Navy  is  having  us 
put  their  designation  on  the  cover.  But  where  there  is  a 
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difference,  (5  or  8),  we  are  putting  supplemental  pages 
in  each  by  processing  a  change - 

Q  For  instance,  if  you  have  a  contract  with  the  Navy  or  the 
Air  Force  using  a  commercial  manual,  part  of  which  would 
be  updates  and  changes,  so  that  as  you  change  that  manual 
and  as  those  changes  go  out  to  your  commercial  customers, 
would  they  also  go  out  to  the  military  so  that  the 
manual  would  also  stay  current? 

A  Yes,  absolutely.  If  it  is  military,  it  must  be  on 

contract  to  cover  that.  There's  a  contract  on  the  F-1C3. 
Sometimes  changes  do  not  occur  on  a  timely  schedule  or 
different  approvals.  It's  difficult  to  keep  the  military 
concurrent  with  the  commercial  because  of  the  procedure 
differences . 

Q  Concerning  the  ATA  series  specifications.  What  are  some 
recent  events  for  the  supplemental  data  base? 

A  1.  Basic  requirements  to  support  a  task  number  manual  is 
five  years  old  and  the  standards  and  requirements  have 
changed. 

2.  Task  force  put  out  new  standards  for  the  data  base 
and  magnetic  tapes  that  goes  out  to  the  airlines. 

3.  Now  working  on  compact  disk  standards.  The  CB  will 
hold  the  major  manuals.  The  stands  will  cover  the 
physical  disk,  what  will  be  seen  on  the  screen,  should 
color  be  used  for  warnings,  what  data  items  the  airlines 
want  to  retrieve,  etc.  The  disk  will  be  able  to  use  the 
disk  from  nay  company  because  it  is  standardized  (all 
companies  will  use  the  same  format.)  The  airlines  will 
be  able  to  select  their  own  equipment  (portable 
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computers)  to  use  with  the  CD's.  The  CD  will  hold  71-72K 
pages  of  texr,  pictures,  and  graphics. 

Q  Will  the  airlines  be  able  to  manipulate  this  data? 

A  Yes.  Part  of  the  standard  will  be  to  alter  the  document 
on  a  floppy  disk.  They  will  be  able  to  tailor  it  to 
their  own  needs. 

Q  Do  you  have  a  recommendation  for  military  contracts  that 
will  use  commercial  equipment  and  commercial  manuals? 

A  The  military  does  not  have  a  cohesive  (continuity)  group. 
Each  branch  of  the  military  wants  to  do  it  their  way  I 
The  military  should  look  at  what  the  co.m.mercial 
world  has  designed.  "When  you  can  take  an  aircraft,  and 
go  to  a  specification  and  find  that  everything  in  that 
aircraft  can  be  identified  with  a  number  that  is  already 
set  up  and  established.  It  is  not  that  much  work  on  the 
various  military  aircraft  to  adapt  the  same  type  of 
numbering  system."  The  aircraft  uses  the  MIDAS  numbering 
system  and  863  specifications. 
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Box  19:  Abstract 


The  objective  of  this  research  was  to  analyze  the 
adequacy  of  using  commercial  manuals  and  to  identify  the 
correct  acquisition  process  to  obtaining  adequate 
commercial  manuals. 

A  queationnaire  was  developed  and  a  survey  was 
accomplished.  The  survey  was  distributed  to  Deputy  Program 
Managers  for  Logistics  (DPML)  and  Air  Logistics  Centers  that 
utilized  commercial  manuals.  The  most  significant  result 
indicated  that  the  commercial  manuals  that  were  determined 
to  be  adequate/  actually  had  not  been  thoroughly  reviewed. 
This  resulted  in  additional  funding  requirements  for  manual 
supplements  and  system  interim  contract  support.  Other 
significant  issues  included:  the  importance  of  early 
involvement  in  the  acquisition  program  phase:  and/  the  DID 
and  the  SOW  as  effective  contract  mediums. 

Recommendations  to  improve  the  effectiveness  of  the 
commercial  manual  review  included: 

1.  Establishing  a  universal  specification  that  would 
be  used  in  all  contracts. 

2.  Establishing  verification  procedures  for  commercial 
manuals  on  equipment  that  is  integrated  into  a  system. 

3.  Establishing  an  internal  technical  writing 
department  that  will  do  the  changes  to  commercial  manuals. 

4.  Establishing  the  use  of  similar  hardware  (CD 
readers)  to  the  industrial  contractors. 


